From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gentoo-user+bounces-198661-garchives=archives.gentoo.org@lists.gentoo.org>
Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by finch.gentoo.org (Postfix) with ESMTPS id 77B5E15808B
for <garchives@archives.gentoo.org>; Tue, 12 Apr 2022 19:28:15 +0000 (UTC)
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id 6DC36E0965;
Tue, 12 Apr 2022 19:28:09 +0000 (UTC)
Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(No client certificate requested)
by pigeon.gentoo.org (Postfix) with ESMTPS id D1E7CE094F
for <gentoo-user@lists.gentoo.org>; Tue, 12 Apr 2022 19:28:08 +0000 (UTC)
Received: by mail-pg1-f181.google.com with SMTP id q142so18132988pgq.9
for <gentoo-user@lists.gentoo.org>; Tue, 12 Apr 2022 12:28:08 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to;
bh=9S9aWWGrZJ9ohlz5/c/mnbQj8RTnrdKa3g/wRCz1Q4I=;
b=0kNSWhuaE6awjZTrcVo2OTT0sNrD/TGN5WYKUmLEmS2WPXVGBySnplc2UdD2oc41Z5
IrsXUQjvpw5S6FozGjBSTkNlyUidgdBS3KFbstVcV+xDysQjOqv+cxxg0ohODueXU883
nWv3BH2e6ZxhsqX2smzlPSsSqDuUQr9bqm86L8FHY/Z8LBfrD6/ilQ07ndne/b9Lt0Kh
6moAPKBCREG8Pz8BDoo5Lh6VhEahbkfYdA/YsTj6hNcOJ+NTDfIX1iUpr18fiadBTv+r
lxD/mLqO7MU9untB4GkV/X/r5WR26glK+rGnnvwlxmzD4VY92ewveYNpz5eduMapSyHe
iBXQ==
X-Gm-Message-State: AOAM531UU9AsB72cZxaJU8YGfXzK5lNg6K0KDkcklkdggtawBBYlreWU
K0jFPQQxUaYQyWRANFyPPnT5Zu2BoNTYoHVyzzGaAR05
X-Google-Smtp-Source: ABdhPJwoeAcGy2Avwj3hP4+oPPwmLxeCaRO2MYrkfn7tqdM5enCzs7v8SpBdiN2fUVTJk5GtKlhEQ5OzKXPueOnZE8s=
X-Received: by 2002:a63:780f:0:b0:386:5d6f:2153 with SMTP id
t15-20020a63780f000000b003865d6f2153mr31105984pgc.555.1649791687329; Tue, 12
Apr 2022 12:28:07 -0700 (PDT)
Precedence: bulk
List-Post: <mailto:gentoo-user@lists.gentoo.org>
List-Help: <mailto:gentoo-user+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-user+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-user+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-user.gentoo.org>
X-BeenThere: gentoo-user@lists.gentoo.org
Reply-to: gentoo-user@lists.gentoo.org
X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply
MIME-Version: 1.0
References: <13443afe-5879-3108-a19a-9457c0b054d1@gmail.com>
<CAGfcS_mJrRYdOZEgT3nVcoUOOAc_aG96Knuf5xaJoR0Fw01=cA@mail.gmail.com> <590a0548-6121-f6b5-cac3-3bb9202f3b70@gmail.com>
In-Reply-To: <590a0548-6121-f6b5-cac3-3bb9202f3b70@gmail.com>
From: Rich Freeman <rich0@gentoo.org>
Date: Tue, 12 Apr 2022 15:27:57 -0400
Message-ID: <CAGfcS_kN8zwDga0S17aDXeFNdm1_rT6Akd9o3KYwTpy-1UnqVQ@mail.gmail.com>
Subject: Re: [gentoo-user] Hard drive error from SMART
To: gentoo-user@lists.gentoo.org
Content-Type: text/plain; charset="UTF-8"
X-Archives-Salt: 2bee5a93-74e0-4818-8b6f-2a857e6b65d2
X-Archives-Hash: c803282af68c7c3a0a701df616ee59da
On Tue, Apr 12, 2022 at 1:08 PM Dale <rdalek1967@gmail.com> wrote:
>
> I remounted the drives and did a backup. For anyone running up on this,
> just in case one of the files got corrupted, I used a little trick to
> see if I can figure out which one may be bad if any. I took my rsync
> commands from my little script and ran them one at a time with --dry-run
> added. If a file was to be updated on the backup that I hadn't changed
> or added, I was going to check into it before updating my backups.
Unless you're using the --checksum option on rsync this isn't likely
to be effective. By default rsync only looks at size and mtime, so it
isn't going to back up a file unless you intentionally changed it. If
data was silently corrupted this wouldn't detect a change at all
without the --checksum option.
Ultimately if you care about silent corruptions you're best off using
a solution that actually achieves this. btrfs, zfs, or something
whipped up with dm-integrity would be best. At a file level you could
store multiple files and hashes, or use a solution like PAR2. Plain
mdadm raid1 will fix issues if the drive detects and reports errors
(the drive typically has a checksum to do this, but it is a black box
and may not always work). The other solutions will reliably detect
and possibly recover errors even if the drive fails to detect them (a
so-called silent error).
Just about all my linux data these days is on a solution that detects
silent errors - zfs or lizardfs. On ssd-based systems where I don't
want to invest in mirroring I still run zfs to detect errors and just
use frequent backups (ssds are small anyway so they're cheap to
frequently back up, especially if they're on zfs where there are
send-based backup scripts for this, and typically this is for OS
drives where things don't change much anyway).
--
Rich