[gentoo-user] is ddrescue this slow?

[gentoo-user] is ddrescue this slow?

[Valmor de Almeida]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

Hello,

I am trying to recover data from a failed drive. An initial attempt with dd
took over 12 hours and it was not even at half of the 80GB damaged drive; so
I quit. Info on the web pointed me to ddrescue

    ddrescue -n /dev/sda /dev/sdc rescued.log

which has taken over 6 hours so far

Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    58656 MB,  errsize:   4408 kB,  current rate:     4096 B/s
   ipos:    58660 MB,   errors:      32,    average rate:    2958 kB/s
   opos:    58660 MB,     time from last successful read:       0 s
Copying non-tried blocks...

The "current rate" varies and sometimes is down to a few B/s!
Is this normal? The drive copied to is a USB external drive.

Thanks for inputs.

--
Valmor

[-- Attachment #2: Type: text/html, Size: 923 bytes --]

[gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

On Sat, Jan 9, 2010 at 3:33 AM, Valmor de Almeida <val.gentoo@gmail.com>wrote:

>
> Hello,
>
> I am trying to recover data from a failed drive. An initial attempt with dd
> took over 12 hours and it was not even at half of the 80GB damaged drive; so
> I quit. Info on the web pointed me to ddrescue
>
>     ddrescue -n /dev/sda /dev/sdc rescued.log
>
> which has taken over 6 hours so far
>
> Press Ctrl-C to interrupt
> Initial status (read from logfile)
> rescued:         0 B,  errsize:       0 B,  errors:       0
> Current status
> rescued:    58656 MB,  errsize:   4408 kB,  current rate:     4096 B/s
>    ipos:    58660 MB,   errors:      32,    average rate:    2958 kB/s
>    opos:    58660 MB,     time from last successful read:       0 s
> Copying non-tried blocks...
>
> The "current rate" varies and sometimes is down to a few B/s!
> Is this normal? The drive copied to is a USB external drive.
>
> Thanks for inputs.
>
> --
> Valmor
>
>
It's about 10 hours now since it started and here it is

Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    58763 MB,  errsize:  22918 kB,  current rate:    1376 kB/s
   ipos:    58786 MB,   errors:      66,    average rate:    1751 kB/s
   opos:    58786 MB,     time from last successful read:       0 s
Copying non-tried blocks...

Sometimes the "current rate" reads 0 B/s for a long time... and "time from
last successful read" can be 8m.

Would any one know whether this is normal?

Thanks,

--
Valmor

[-- Attachment #2: Type: text/html, Size: 1994 bytes --]

Re: [gentoo-user] Re: is ddrescue this slow?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:

> Sometimes the "current rate" reads 0 B/s for a long time... and "time
> from last successful read" can be 8m.
> 
> Would any one know whether this is normal?

Doesn't ddrescue retry on blocks it cannot read? That would explain the
variable read rate, even the period of zero activity. If your drive is
that badly damaged, dd would have been no use anyway.


-- 
Neil Bothwick

There's too much blood in my caffeine system.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Re: is ddrescue this slow?

[Stroller]

On 9 Jan 2010, at 09:23, Neil Bothwick wrote:
> On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
>
>> Sometimes the "current rate" reads 0 B/s for a long time... and "time
>> from last successful read" can be 8m.
>>
>> Would any one know whether this is normal?
>
> Doesn't ddrescue retry on blocks it cannot read? That would explain  
> the
> variable read rate, even the period of zero activity. If your drive is
> that badly damaged, dd would have been no use anyway.

I think Valmor is using GNU ddrescue, with which one makes the  
multiple passes manually. The "-n" flag on the command line that  
Valmor posted (`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates to  
the examples given in the GNU manual page [1]. I believe that GNU  
ddrescue is the better version - it was inspired by garloff's original  
work, and makes improvements, but it operates differently.

Having said that, it could just be that the drive _firmware) is making  
multiple attempts to read the failing blocks before returning the  
failure (or the data, in the case that a 2nd attempt to read the drive  
was successful) to the host o/s. Isn't this how hard-drives work?

ddrescue worked fast here when I tried it here recently on a drive  
with only one unreadable block, but Valmor's drive is failing much  
more severely. TBH, I would expect reads from a badly-failing drive,  
but this is an intuitive expectation, not a reasoned one. I think the  
best thing he can do is hold his breath, wait until its finished and  
see how if the results are readable, after running `fsck` on the  
mounted filesystem.

Valmor: when I ran the `ddrescue -dr3` stage I had no success at all,  
however the system was fine after a reboot & a `chkdsk`. Better than  
it had been, in fact, on the old hard-drive. You might have more luck  
getting *some* of the blocks showing as failed when you run it on your  
drive, but don't be too disheartened if you don't.

Stroller.




[1] http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Examples

[gentoo-user] Re: is ddrescue this slow?

[Grant Edwards]

On 2010-01-09, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
>
>> Sometimes the "current rate" reads 0 B/s for a long time... and "time
>> from last successful read" can be 8m.
>> 
>> Would any one know whether this is normal?
>
> Doesn't ddrescue retry on blocks it cannot read?
>
> That would explain the variable read rate, even the period of
> zero activity. If your drive is that badly damaged, dd would
> have been no use anyway.

Yes.  On a disk with a lot of bad blocks it can be very slow as
it retries bad blocks and narrows the failed regions.

-- 
Grant

Re: [gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

[-- Attachment #1: Type: text/plain, Size: 4027 bytes --]

On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stroller@stellar.eclipse.co.uk>wrote:

>
> On 9 Jan 2010, at 09:23, Neil Bothwick wrote:
>
>> On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
>>
>>  Sometimes the "current rate" reads 0 B/s for a long time... and "time
>>> from last successful read" can be 8m.
>>>
>>> Would any one know whether this is normal?
>>>
>>
>> Doesn't ddrescue retry on blocks it cannot read? That would explain the
>> variable read rate, even the period of zero activity. If your drive is
>> that badly damaged, dd would have been no use anyway.
>>
>
> I think Valmor is using GNU ddrescue, with which one makes the multiple
> passes manually. The "-n" flag on the command line that Valmor posted
> (`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates to the examples given
> in the GNU manual page [1]. I believe that GNU ddrescue is the better
> version - it was inspired by garloff's original work, and makes
> improvements, but it operates differently.
>
>
Indeed I am using GNU ddrescue and the -n flag is supposed to expedite the
recovery of data as posted in
http://www.cgsecurity.org/wiki/Damaged_Hard_Disk

"The best solution - both faster and more efficient - seems to be Antonio
Diaz's 'ddrescue' (ddrescue <http://savannah.gnu.org/projects/ddrescue/>)"

# first, grab most of the error-free areas in a hurry:
./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
# then try to recover as much of the dicy areas as possible:
./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log


expectation, not a reasoned one. I think the best thing he can do is hold
> his breath, wait until its finished and see how if the results are readable,
> after running `fsck` on the mounted filesystem.
>

The first step above finished; don't know how long it took but it was a long
time (maybe 20 hours or more?) and the screen output was

Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    58811 MB,  errsize:  48909 kB,  current rate:       83 B/s
   ipos:    58860 MB,   errors:      95,    average rate:    1365 kB/s
   opos:    58860 MB,     time from last successful read:       0 s
Copying non-tried blocks...
ddrescue: write error: Input/output error


Comparing with the screen output at the time of my first post,

Current status rescued went from 58656 MB to 58811 MB, errsize went from
4408 kB to 48909 kB.

Don't know how the write error: Input/output error message affect the data
in the new drive copied to. Not sure whether I should do the next step with
option -r 1.

This failed drive is still bootable and the corruption is in the partitions
/var (which I do not care) and /home; these cannot be mounted. I would like
to attempt to get a couple of files from /home that were not in the most
recent backup. Maybe I should try to rescue only the partition /home.
However this partition is under LVM. Specifically, /dev/sda4 is a linux LVM
partition. The volume group is vfda and the logical volume of interest is
/dev/vfda/home which has reiserfs file system. Is it possible to rescue data
only from this partition when under LVM?



>
> Valmor: when I ran the `ddrescue -dr3` stage I had no success at all,
> however the system was fine after a reboot & a `chkdsk`. Better than it had
> been, in fact, on the old hard-drive. You might have more luck getting
> *some* of the blocks showing as failed when you run it on your drive, but
> don't be too disheartened if you don't.
>
> Stroller.
>
>
>
Stroller, you mean your rescue.log showed no problematic entries? I got over
400 lines in my rescue.log file.

root@sysresccd /root % head rescued.log
# Rescue Logfile. Created by GNU ddrescue version 1.11
# current_pos  current_status
0xDB45D9000     ?
#      pos        size  status
0x00000000  0x9CE341000  +
0x9CE341000  0x00000200  -
0x9CE341200  0x0001F000  *
0x9CE360200  0x00000200  -
0x9CE360400  0x00020000  *
0x9CE380400  0x3BD63AC00  +



Thanks for inputs.

--
Valmor

>
>

[-- Attachment #2: Type: text/html, Size: 5539 bytes --]

Re: [gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stroller@stellar.eclipse.co.uk>wrote:

[snip]


>  in the GNU manual page [1]. I believe that GNU ddrescue is the better
> version - it was inspired by garloff's original work, and makes
> improvements, but it operates differently.
>

Comment. Another reason I moved away from dd (apart from the slow running
time) to ddrescue was because of this note related to LVM.

http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html
"Steve Holmes reports that dd with conv=sync,noerror doesn't correctly image
disks with LVM2 Logical Volumes. I haven't investigated this. He also points
out GNU ddrescue <http://www.gnu.org/software/ddrescue/ddrescue.html> ( not
the same as dd_rescue mentioned above) which looks useful. According to
Steve, ddrescue works finewith LVM2, and some
people<http://www.mail-archive.com/bug-ddrescue@gnu.org/msg00038.html>seem
to suggest it's generally superior to dd_rescue."

The partition I would like to get data from is under LVM (previous post).

Thanks,

--
Valmor

[-- Attachment #2: Type: text/html, Size: 1572 bytes --]

Re: [gentoo-user] Re: is ddrescue this slow?

[Stroller]

[-- Attachment #1: Type: text/plain, Size: 2455 bytes --]

Would love to comment on this. Is it possible you could resend this  
post in plain text format?

Stroller.


On 10 Jan 2010, at 02:08, Valmor de Almeida wrote:

> On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stroller@stellar.eclipse.co.uk 
> > wrote:
>
> On 9 Jan 2010, at 09:23, Neil Bothwick wrote:
> On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
>
> Sometimes the "current rate" reads 0 B/s for a long time... and "time
> from last successful read" can be 8m.
>
> Would any one know whether this is normal?
>
> Doesn't ddrescue retry on blocks it cannot read? That would explain  
> the
> variable read rate, even the period of zero activity. If your drive is
> that badly damaged, dd would have been no use anyway.
>
> I think Valmor is using GNU ddrescue, with which one makes the  
> multiple passes manually. The "-n" flag on the command line that  
> Valmor posted (`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates  
> to the examples given in the GNU manual page [1]. I believe that GNU  
> ddrescue is the better version - it was inspired by garloff's  
> original work, and makes improvements, but it operates differently.
>
>
> Indeed I am using GNU ddrescue and the -n flag is supposed to  
> expedite the recovery of data as posted in http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
> "The best solution - both faster and more efficient - seems to be  
> Antonio Diaz's 'ddrescue' (ddrescue)"
>
> # first, grab most of the error-free areas in a hurry:
> ./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
> # then try to recover as much of the dicy areas as possible:
> ./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log
>
>
> expectation, not a reasoned one. I think the best thing he can do is  
> hold his breath, wait until its finished and see how if the results  
> are readable, after running `fsck` on the mounted filesystem.
>
> The first step above finished; don't know how long it took but it  
> was a long time (maybe 20 hours or more?) and the screen output was
>
> Press Ctrl-C to interrupt
> Initial status (read from logfile)
> rescued:         0 B,  errsize:       0 B,  errors:       0
> Current status
> rescued:    58811 MB,  errsize:  48909 kB,  current rate:       83 B/s
>    ipos:    58860 MB,   errors:      95,    average rate:    1365 kB/s
>    opos:    58860 MB,     time from last successful read:       0 s
> Copying non-tried blocks...
> ddrescue: write error: Input/output error
>


[-- Attachment #2: Type: text/html, Size: 4033 bytes --]

Re: [gentoo-user] Re: is ddrescue this slow?

[Dale]

This help?  Should be plain text. 

Dale

:-)  :-)

Stroller wrote:
> Would love to comment on this. Is it possible you could resend this 
> post in plain text format?
>
> Stroller.
>
>
> On 10 Jan 2010, at 02:08, Valmor de Almeida wrote:
>
>>
>> Indeed I am using GNU ddrescue and the -n flag is supposed to 
>> expedite the recovery of data as posted in 
>> http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
>>
>> "The best solution - both faster and more efficient - seems to be 
>> Antonio Diaz's 'ddrescue' (ddrescue 
>> <http://savannah.gnu.org/projects/ddrescue/>)"
>>
>> # first, grab most of the error-free areas in a hurry:
>> ./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
>> # then try to recover as much of the dicy areas as possible:
>> ./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log
>>
>>     
>>
>>     expectation, not a reasoned one. I think the best thing he can do
>>     is hold his breath, wait until its finished and see how if the
>>     results are readable, after running `fsck` on the mounted filesystem.
>>
>>
>> The first step above finished; don't know how long it took but it was 
>> a long time (maybe 20 hours or more?) and the screen output was
>>
>> Press Ctrl-C to interrupt
>> Initial status (read from logfile)
>> rescued:         0 B,  errsize:       0 B,  errors:       0
>> Current status
>> rescued:    58811 MB,  errsize:  48909 kB,  current rate:       83 B/s
>>    ipos:    58860 MB,   errors:      95,    average rate:    1365 kB/s
>>    opos:    58860 MB,     time from last successful read:       0 s
>> Copying non-tried blocks...
>> ddrescue: write error: Input/output error
>>
>

Re: [gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]

On Sun, Jan 10, 2010 at 8:09 AM, Stroller <stroller@stellar.eclipse.co.uk>wrote:

> Would love to comment on this. Is it possible you could resend this post in
> plain text format?
>
> Stroller.
>
>
Below is my last post copied and pasted into gmail without the html hot
links. I am doing this from within systemrescuecd using firefox and gmail.
Don't know how to make it plain ascii otherwise. Hopefully just eliminating
the html links will work. Stroller, is this what you are referring to?

Thanks,

--
Valmor


>
> On Sat, Jan 9, 2010 at 11:10 AM, Stroller  wrote:
>
> [snip]
>
>
>>  in the GNU manual page [1]. I believe that GNU ddrescue is the better
>> version - it was inspired by garloff's original work, and makes
>> improvements, but it operates differently.
>>
>
> Comment. Another reason I moved away from dd (apart from the slow running
> time) to ddrescue was because of this note related to LVM.
>
> h<http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html>
>

http://www.interference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html

> "Steve Holmes reports that dd with conv=sync,noerror doesn't correctly
> image disks with LVM2 Logical Volumes. I haven't investigated this. He also
> points out GNU ddrescue ( not the same as dd_rescue mentioned above) which
> looks useful. According to Steve, ddrescue works finewith LVM2, and  some
> people seem to suggest it's generally superior to dd_rescue."
>
> The partition I would like to get data from is under LVM (previous post).
>
> Thanks,
>
> --
> Valmor
>
>

[-- Attachment #2: Type: text/html, Size: 3003 bytes --]

Re: [gentoo-user] Re: is ddrescue this slow?

[Stroller]

On 10 Jan 2010, at 18:09, Valmor de Almeida wrote:

> On Sun, Jan 10, 2010 at 8:09 AM, Stroller <stroller@stellar.eclipse.co.uk 
> > wrote:
>> Would love to comment on this. Is it possible you could resend this  
>> post in plain text format?
>>
>> Stroller.
>
>
> Below is my last post copied and pasted into gmail without the html  
> hot links. I am doing this from within systemrescuecd using firefox  
> and gmail. Don't know how to make it plain ascii otherwise.  
> Hopefully just eliminating the html links will work. Stroller, is  
> this what you are referring to?

Both messages contain html text formatting. If you look at your last  
message (the one to which I'm replying now, Message-ID: <128ccc221001101009v75f23dcey7d52967b16f7d13b@mail.gmail.com 
 >) in a text editor, for instance:

<div><div></div>
<div class=3D"h5"><div><br></div></div></div></div></ 
blockquote><div><br>Be=
low is my last post copied and pasted into gmail without the html hot  
links=
. I am doing this from within systemrescuecd using firefox and gmail.  
Don&#=
39;t know how to make it plain ascii otherwise. Hopefully just  
eliminating =
the html links will work. Stroller, is this what you are referring to?  
<br>
<br>Thanks,<br><br>--<br>Valmor<br>=A0<br></div><blockquote  
class=3D"gmail_=
quote" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt  
0pt =
0pt 0.8ex; padding-left: 1ex;"><div style=3D"word-wrap: break- 
word;"><div><=
div class=3D"h5">
<div></div><br><div class=3D"im">On Sat, Jan 9, 2010 at 11:10 AM,  
Stroller=
=A0<span dir=3D"ltr"></span> wrote:<br></div><div  
class=3D"gmail_quote"><di=
v><br>[snip]<br>=A0<br></div><div class=3D"im"><blockquote  
class=3D"gmail_q=
uote" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt  
0pt 0=
pt 0.8ex; padding-left: 1ex;">

The problem I have with this is that it seems difficult in my mail  
client to snip excessive quoting when I reply. If I switch to plain  
text, all the quoting marks / indenting disappears because (as per the  
snippet above) it's using an arbitrary html construction ("border- 
left: 1px solid; margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> to  
represent that.

Just use plain text. It's the convention. It's simple, it works. If  
Gmail imposes this html upon you then I suggest you use IMAP to access  
your gmail account. You're using Gentoo, so there are plenty of  
clients which enable you to do this: mutt, pine, Thunderbird,  
Kmail, ... any of them can post in plain text.

I'm sorry to sound like a knob - I do admit to having a bee in my  
bonnet about this at the moment. I have a client who sends me 100kb  
emails with tiny unreadable text, a yellow background and multiple  
images (at least 7), just to convey a sentence or two of actual  
content. They pad maybe 150 bytes of text with 100,000 bytes of crap  
(the entire text of "Pride & Prejudice" is 700kb), and its effect is  
to make the content less readable; I can't bitch to them about their  
idiocy, because they pay me thousands a year.

I know this isn't your fault, but I had been quite busy for some hours  
and was perhaps tired when I clicked reply to your message last night.  
I clicked on the text so as to quote inline and a whole paragraph lit  
up as my mailer tried to deal with it; so I clicked on y mailer's  
plain text button and all the quote marks disappeared and I couldn't  
see who had said what. I just couldn't be arsed to deal with it. Just  
use plain text. It's the convention. It's simple, it works.

Stroller.

Re: [gentoo-user] Re: is ddrescue this slow?

[Stroller]

On 10 Jan 2010, at 21:28, Stroller wrote:
> On 10 Jan 2010, at 18:09, Valmor de Almeida wrote:
>> ...  I am doing this from within systemrescuecd using firefox and  
>> gmail. Don't know how to make it plain ascii otherwise. Hopefully  
>> just eliminating the html links will work. Stroller, is this what  
>> you are referring to?
>
> ... If Gmail imposes this html upon you then I suggest you use IMAP  
> to access your gmail account. You're using Gentoo, so there are  
> plenty of clients which enable you to do this: mutt, pine,  
> Thunderbird, Kmail, ... any of them can post in plain text.

I just reread the above ("from within systemrescuecd"), which implies  
you may be stuck without another working system.

Google says "If you decide you'd like to write a message in plain text  
format, just click << Plain text along the top of the compose window."  
<http://mail.google.com/support/bin/answer.py?hl=en&answer=8260>

If you're stuck without another working system and this doesn't work  
then let me know & I'll make the extra effort, with apologies (and  
with sympathies for your difficult working conditions - I am sorry if  
I have exacerbated them).

Stroller.

Re: [gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

On Sun, Jan 10, 2010 at 9:28 PM, Stroller
<stroller@stellar.eclipse.co.uk> wrote:
>
> On 10 Jan 2010, at 18:09, Valmor de Almeida wrote:
>
>> On Sun, Jan 10, 2010 at 8:09 AM, Stroller <stroller@stellar.eclipse.co.uk>
>> wrote:
>>>
>>> Would love to comment on this. Is it possible you could resend this post
>>> in plain text format?
>>>
>>> Stroller.
>>
>>
>> Below is my last post copied and pasted into gmail without the html hot
[snip]
>
> Both messages contain html text formatting. If you look at your last message
> (the one to which I'm replying now, Message-ID:
> <128ccc221001101009v75f23dcey7d52967b16f7d13b@mail.gmail.com>) in a text
> editor, for instance:

I share all of your comments. Yes I was/am stuck inside
systemrescuecd. I typically use thunderbird to get my mail from the
gmail server as imap and always use plain text (to send and receive).
Therefore I am not knowledgeable of the web gmail application; I
seldom log into my gmail account with a web browser. Your next e-mail
pointed me to the "plain" option; thanks! Don't know how I could have
missed it. Originally I had looked at the settings of my gmail account
but it did not help. On my next e-mail I will send a clean plain text
resend.

Thanks,

--
Valmor

Re: [gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

On Sun, Jan 10, 2010 at 8:09 AM, Stroller
<stroller@stellar.eclipse.co.uk> wrote:
> Would love to comment on this. Is it possible you could resend this post in
> plain text format?
> Stroller.

Here it goes.

>On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stroller@stellar.eclipse.co.uk> >wrote:

[snip]

>    in the GNU manual page [1]. I believe that GNU ddrescue is the better >version - it was inspired by garloff's original work, and makes improvements, >but it operates differently.

Comment. Another reason I moved away from dd (apart from the slow
running time) to ddrescue was because of this note related to LVM.

http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html
"Steve Holmes reports that dd with conv=sync,noerror doesn't correctly
image disks with LVM2 Logical Volumes. I haven't investigated this. He
also points out GNU ddrescue ( not the same as dd_rescue mentioned
above) which looks useful. According to Steve, ddrescue works finewith
LVM2, and some people seem to suggest it's generally superior to
dd_rescue."

The partition I would like to get data from is under LVM (previous post).

Thanks,

--
Valmor

Re: [gentoo-user] Re: is ddrescue this slow?

[Valmor de Almeida]

On Sun, Jan 10, 2010 at 9:39 PM, Stroller
<stroller@stellar.eclipse.co.uk> wrote:
>
[snip]
>
> I just reread the above ("from within systemrescuecd"), which implies you
> may be stuck without another working system.

Yes.
I am afraid the text below will be confusing but will send anyway.
Will be happy to send clarifications if needed. This is a resend (in
plain text) of another e-mail I sent with information on the LVM
partition on the broken drive.

>On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stroller@stellar.eclipse.co.uk> >wrote:
[snip]
>    I think Valmor is using GNU ddrescue, with which one makes the multiple >passes manually. The "-n" flag on the command line that Valmor posted >(`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates to the examples given >in the GNU manual page [1]. I believe that GNU ddrescue is the better version >- it was inspired by garloff's original work, and makes improvements, but it >operates differently.


Indeed I am using GNU ddrescue and the -n flag is supposed to expedite
the recovery of data as posted in
http://www.cgsecurity.org/wiki/Damaged_Hard_Disk

"The best solution - both faster and more efficient - seems to be
Antonio Diaz's 'ddrescue' (ddrescue)"

# first, grab most of the error-free areas in a hurry:
./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
# then try to recover as much of the dicy areas as possible:
./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log


>    expectation, not a reasoned one. I think the best thing he can do is hold >his breath, wait until its finished and see how if the results are readable, after >running `fsck` on the mounted filesystem.


The first step above finished; don't know how long it took but it was
a long time (maybe 20 hours or more?) and the screen output was


Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    58811 MB,  errsize:  48909 kB,  current rate:       83 B/s
   ipos:    58860 MB,   errors:      95,    average rate:    1365 kB/s
   opos:    58860 MB,     time from last successful read:       0 s
Copying non-tried blocks...
ddrescue: write error: Input/output error


Comparing with the screen output at the time of my first post,

Current status rescued went from 58656 MB to 58811 MB, errsize went
from 4408 kB to 48909 kB.

Don't know how the write error: Input/output error message affect the
data in the new drive copied to. Not sure whether I should do the next
step with option -r 1.

This failed drive is still bootable and the corruption is in the
partitions /var (which I do not care) and /home; these cannot be
mounted. I would like to attempt to get a couple of files from /home
that were not in the most recent backup. Maybe I should try to rescue
only the partition /home. However this partition is under LVM.
Specifically, /dev/sda4 is a linux LVM partition. The volume group is
vfda and the logical volume of interest is /dev/vfda/home which has
reiserfs file system. Is it possible to rescue data only from this
partition when under LVM?

>    Valmor: when I ran the `ddrescue -dr3` stage I had no success at all, >however the system was fine after a reboot & a `chkdsk`. Better than it had >been, in fact, on the old hard-drive. You might have more luck getting *some* >of the blocks showing as failed when you run it on your drive, but don't be >too disheartened if you don't.

>   Stroller.

Stroller, you mean your rescue.log showed no problematic entries? I
got over 400 lines in my rescue.log file.

root@sysresccd /root % head rescued.log
# Rescue Logfile. Created by GNU ddrescue version 1.11
# current_pos  current_status
0xDB45D9000     ?
#      pos        size  status
0x00000000  0x9CE341000  +
0x9CE341000  0x00000200  -
0x9CE341200  0x0001F000  *
0x9CE360200  0x00000200  -
0x9CE360400  0x00020000  *
0x9CE380400  0x3BD63AC00  +



Thanks for inputs.

--
Valmor

Re: [gentoo-user] Re: is ddrescue this slow?

[Stroller]

On 11 Jan 2010, at 00:14, Valmor de Almeida wrote:
> ...
> Indeed I am using GNU ddrescue and the -n flag is supposed to expedite
> the recovery of data as posted in
> http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
>
> "The best solution - both faster and more efficient - seems to be
> Antonio Diaz's 'ddrescue' (ddrescue)"
>
> # first, grab most of the error-free areas in a hurry:
> ./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
> # then try to recover as much of the dicy areas as possible:
> ./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log

I remember reading a very similar summary - the words "grab most of  
the error-free areas in a hurry" seem quite familiar - when I started  
using ddrescue, a few months ago.

> The first step above finished; don't know how long it took but it was
> a long time (maybe 20 hours or more?) and the screen output was
>
>
> Press Ctrl-C to interrupt
> Initial status (read from logfile)
> rescued:         0 B,  errsize:       0 B,  errors:       0
> Current status
> rescued:    58811 MB,  errsize:  48909 kB,  current rate:       83 B/s
>   ipos:    58860 MB,   errors:      95,    average rate:    1365 kB/s
>   opos:    58860 MB,     time from last successful read:       0 s
> Copying non-tried blocks...
> ddrescue: write error: Input/output error
>
>
> Comparing with the screen output at the time of my first post,
>
> Current status rescued went from 58656 MB to 58811 MB, errsize went
> from 4408 kB to 48909 kB.
>
> Don't know how the write error: Input/output error message affect the
> data in the new drive copied to. Not sure whether I should do the next
> step with option -r 1.
>
> This failed drive is still bootable and the corruption is in the
> partitions /var (which I do not care) and /home; these cannot be
> mounted. I would like to attempt to get a couple of files from /home
> that were not in the most recent backup. Maybe I should try to rescue
> only the partition /home. However this partition is under LVM.
> Specifically, /dev/sda4 is a linux LVM partition. The volume group is
> vfda and the logical volume of interest is /dev/vfda/home which has
> reiserfs file system. Is it possible to rescue data only from this
> partition when under LVM?

I would try running fsck on a copy of the image.

Your original command was to clone to /dev/sdc - you can now safely  
`dd if=/dev/sdc2 of=/mnt/foo/olddrive.img` and then loopback mount the  
image. You should be able to fun fsck on it, also.  (Where sdc2 is the  
partition /home was on)

I have invested in a fair amount of hard-drive space recently, and the  
extra room allows you a lot of breathing space for operations like this.

If fsck doesn't work then you can dry photorec. I would try on both  
sdcX and olddrive.img (where X is the partition /home was on).

DISCLAIMER: I don't know anything about LVM or how that might affect  
things. In fact, I avoid LVM for this reason. I might do something  
like backing up (using `dd`) the other partitions that contribute to  
the affected volume groups, then lvm-mount them together with your  
sdcX and try an reiserfsck on the resultant logical volume. The idea  
position to be in (well, "ideal" once we have accepted the fact of the  
failed drive) is to have images of all your drives before you start  
mucking about any further; if the repair attempt fails you can revert  
to the images and try again a different way.


>>   Valmor: when I ran the `ddrescue -dr3` stage I had no success at  
>> all, however the system was fine after a reboot & a `chkdsk`.  
>> Better than it had been, in fact, on the old hard-drive. You might  
>> have more luck getting *some* of the blocks showing as failed when  
>> you run it on your drive, but don't be too disheartened if you don't.
>
> Stroller, you mean your rescue.log showed no problematic entries?

No, I had only one line in my rescue.log, my most recent attempt at  
this. The `ddrescue -dr3` stage errored without managing to recover  
that block, but it seems like the data in that block was not really  
important.

The old drive failed to completely boot Windows because of this bad  
block; the new drive - with that block missing - booted fine (perhaps  
after a chkdsk). So I can only conclude that - using the old drive -  
Windows died because some unimportant system file was thought to be  
there. It couldn't read the block and kept trying. With the new drive  
the file was clearly absent or damaged, so Windows was able to  
continue the boot without it. Perhaps some unimportant system service  
failed to start, but in any case the system worked fine.

I'm pretty sure I also dealt last year with another drive which had  
about 3 errors on it, but I don't remember any of the details.

> I got over 400 lines in my rescue.log file.
>
> root@sysresccd /root % head rescued.log
> # Rescue Logfile. Created by GNU ddrescue version 1.11
> # current_pos  current_status
> 0xDB45D9000     ?
> #      pos        size  status
> 0x00000000  0x9CE341000  +
> 0x9CE341000  0x00000200  -
> 0x9CE341200  0x0001F000  *
> 0x9CE360200  0x00000200  -
> 0x9CE360400  0x00020000  *
> 0x9CE380400  0x3BD63AC00  +


It seems to me that your drive was in a very bad way, and this is why  
you has so many errors. You don't say how large it was - ddrescue  
terminated at  58811 MB. Was it only a 60GB drive?

It seems to me that ddrecue died before managing to complete the first  
pass. I don't think you should have seen the "ddrescue: write error:  
Input/output error" line if the first pass completed ok - you should  
have just seen something like "Completed: XXX copied, Y errors". Then  
you would try a second pass (eg `ddrescue -r 3`) and *hope* to get  
fewer errors the second time around; the -r 3 tells ddrescue to keep  
trying, make 3 attempts to read this block before giving up. Because  
the logfile keeps a record of the failed blocks, when you run ddrescue  
with the -r flag it only tries the blocks that have previously failed.  
So you can quite safely keep running `ddrescue -r 3 /dev/old_disk /dev/ 
new_disk rescued.log`, optimistically hoping that maybe this time the  
read will be successful. The only important thing is to make sure you  
always use the same destination and rescued.log file when doing so -  
using the same rescued.log file ensures you're not wasting time trying  
blocks that you've already got on a previous pass, and using the same  
destination ensures that any blocks that are read successfully this  
time are written into an image file that is combined from the multiple  
passes.

I think that the "write error: Input/output error" line indicates that  
your drive really has failed badly & catastrophically, and that you're  
not going to get any more off it. Feel free to try though. Console  
yourself with the knowledge that ddrescue was the best tool for the  
job, and that in using it you did all you could have done to recover  
your data.

The number of errors you saw (95) indicates your drive was much worse  
than the ones I have recently worked on. I'm pretty sure my  
rescued.log was less than 40 lines long (I could `cat` or `less` it,  
and it fitted within my 80 wide x 50 tall terminal, I'm sure).

I hope this makes sense. I'm by no means an expert, but I'm glad to  
help in any way possible. Having lots of disk space helps a lot.

Stroller.

Re: [gentoo-user] [CLOSED] Re: is ddrescue this slow?

[Valmor de Almeida]

Stroller wrote:
[snip]
> 
> I would try running fsck on a copy of the image.
> 

I did try and thought I would post here some final info just for the
record. I proceeeded with the command

  root@sysresccd /root % ddrescue -r 1 /dev/sda /dev/sdc rescued.log

which took ~50 hours to finish with the message:

Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued:    58811 MB,  errsize:  48909 kB,  errors:      95
Current status
rescued:    77815 MB,  errsize:   2210 MB,  current rate:        0 B/s
   ipos:    66589 MB,   errors:     598,    average rate:    56872 B/s
   opos:    66589 MB,     time from last successful read:    18.5 m
Trimming failed blocks...
ddrescue: write error: Input/output error

Again the same error message. I did not care about it and moved forward
to mount the /dev/sdc drive. Enabled the LVM volume groups with vgchange
-a y (all this under the systemrescuecd boot), mounted the partition of
interest under LVM control and did a

  reiserfsck --check  /dev/myvg/mylv

It ended with

 1 found corruptions can be fixed only when running with --rebuild-tree
 ###########
 reiserfsck finished at Fri Jan 15 11:46:23 2010
 ###########

The next step was then

  reiserfsck --rebuild-tree --logfile rebuild.log /dev/myvg/mylv

I was then able to mount the partition and inspect the newly created
lost+found/ directory. Surprisingly I was able to find the file I was
looking for!!

It was the first time I tried this kind of HDD forensics and was
surprised with the time that it took to recover data from a relative low
storage drive: 80GB. The rebuild.log file had over 8000 lines.

Stroller, thanks for all your comments and suggestions. Yes having extra
disk space is a must to be able to recover data.

--
Valmor

> 
> I hope this makes sense. I'm by no means an expert, but I'm glad to  
> help in any way possible. Having lots of disk space helps a lot.
> 
> Stroller.
> 
> 
>