From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 3E52D13837C for ; Tue, 8 Jan 2013 22:17:30 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 85A4521C041; Tue, 8 Jan 2013 22:17:10 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B892821C022 for ; Tue, 8 Jan 2013 22:15:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id E3D7533DA35 for ; Tue, 8 Jan 2013 22:15:39 +0000 (UTC) X-Virus-Scanned: by amavisd-new using ClamAV at gentoo.org X-Spam-Flag: NO X-Spam-Score: -1.282 X-Spam-Level: X-Spam-Status: No, score=-1.282 tagged_above=-999 required=5.5 tests=[AWL=-2.483, DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001, NML_ADSP_CUSTOM_MED=1.2, RP_MATCHES_RCVD=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=no Received: from smtp.gentoo.org ([IPv6:::ffff:127.0.0.1]) by localhost (smtp.gentoo.org [IPv6:::ffff:127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kS8ONOvRTj7s for ; Tue, 8 Jan 2013 22:15:34 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 62C0733DA7B for ; Tue, 8 Jan 2013 22:15:34 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1TshSQ-0002pH-R2 for gentoo-user@gentoo.org; Tue, 08 Jan 2013 23:15:42 +0100 Received: from dsl.comtrol.com ([64.122.56.22]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Jan 2013 23:15:42 +0100 Received: from grant.b.edwards by dsl.comtrol.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Jan 2013 23:15:42 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-user@lists.gentoo.org From: Grant Edwards Subject: [gentoo-user] Re: OT: Fighting bit rot Date: Tue, 8 Jan 2013 22:15:15 +0000 (UTC) Message-ID: References: <50EB2BF7.4040109@binarywings.net> <20130108012016.2f02c68c@khamul.example.com> <50EBCA77.8030603@binarywings.net> <20130108095510.04f84040@khamul.example.com> <50EC4660.5090208@binarywings.net> <20130108234504.08c19c9c@khamul.example.com> X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: dsl.comtrol.com User-Agent: slrn/pre1.0.0-18 (Linux) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Archives-Salt: 2709c176-fd9e-419d-8971-b3dbc17f5dcc X-Archives-Hash: 146357296e83f6806c31b81c8ecd6b49 On 2013-01-08, Alan McKinnon wrote: >> When a hard drive starts to fail, you don't unknowingly get back >> "rotten" data with some bits flipped. You get either a "seek error" >> or "read error", and no data at all. IIRC, the same is true for >> attempts to read a failing CD. > > I see what Florian is getting at here, and he's perfectly correct. > > We techie types often like to think our storage is purely binary, the > cells are either on or off and they never change unless we > deliberately make them change. We think this way because we wrap our > storage in layers to make it look that way, in the style of an API. > > The truth is that our storage is subject to decay. Harddrives are > magnetic at heart, and atoms have to align and stay aligned for the > drive to work. Floppies are infinitely worse at this, but drives are > not immune. Writeable CDs do not have physical pits and lands like > factory original discs have, they use chemicals to make reflective and > non-reflective spots. The list of points of corruption is long and > they all happen after the data has been committed to physical storage. True. But, in my experience, the chances of any of those failures resulting in a successful read of incorrect data is vanishly small. > Worse, you only know about the corruption by reading it, there is no > other way to discover if the medium and the data are still OK. He > wants to read the medium occasionally That may be a good idea, and will detect media failures. > and verify it That's the part I think is pointless in practice (if you're trying to detect failing media). > while the backups are still usable, and not wait for the point of no > return - the "read error" from a medium that long since failed. My point is that _comparing_data_to_a_backup_ just isn't a useful, practical way to detect failing hard drives, optical drives, or CDs. I've seen a lot of hard drives, optical drives, floppy drives, flopies, and CDs fail. The failure mode in every case has been a "seek error" or "read error" resulting in _no_data_ rather than a read returning erroneous data. It seems that in laboratory conditions, people have managed to see erroneous data, but I'm not convinced worrying about it is worthwhile. IMO, having backup data _is_ very valuable, but regularly reading files and comparing them to backup copies isn't a useful way to detect failing media. You're much more likely to detect failing RAM (which is useful, but there are better ways to do it). -- Grant Edwards grant.b.edwards Yow! I think I am an at overnight sensation right gmail.com now!!