From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 7EEF61389ED for ; Tue, 28 Oct 2014 00:56:15 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id B064EE08F9; Tue, 28 Oct 2014 00:56:08 +0000 (UTC) Received: from sempidan.tirtonadi.com (198.11.216.210-static.reverse.softlayer.com [198.11.216.210]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 74741E08CE for ; Tue, 28 Oct 2014 00:56:07 +0000 (UTC) Received: from mail-la0-f52.google.com ([209.85.215.52]:54935) by sempidan.tirtonadi.com with esmtpsa (TLSv1:RC4-SHA:128) (Exim 4.82) (envelope-from ) id 1Xiv4w-0008T1-3K for gentoo-user@lists.gentoo.org; Tue, 28 Oct 2014 07:56:06 +0700 Received: by mail-la0-f52.google.com with SMTP id hz20so6749036lab.25 for ; Mon, 27 Oct 2014 17:56:01 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 X-Received: by 10.112.221.226 with SMTP id qh2mr13037lbc.5.1414457761332; Mon, 27 Oct 2014 17:56:01 -0700 (PDT) Received: by 10.152.4.7 with HTTP; Mon, 27 Oct 2014 17:56:01 -0700 (PDT) Received: by 10.152.4.7 with HTTP; Mon, 27 Oct 2014 17:56:01 -0700 (PDT) In-Reply-To: References: <201410270924.40381.michaelkintzios@gmail.com> <544E2875.5000309@gmail.com> <201410271522.32452.michaelkintzios@gmail.com> <544E7F83.4020303@googlemail.com> Date: Tue, 28 Oct 2014 07:56:01 +0700 Message-ID: Subject: Re: [gentoo-user] Safeguarding strategies against SSD data loss From: Pandu Poluan To: gentoo-user@lists.gentoo.org Content-Type: multipart/alternative; boundary=001a1135ecb86a03ed0506711cef X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sempidan.tirtonadi.com X-AntiAbuse: Original Domain - lists.gentoo.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - poluan.info X-Get-Message-Sender-Via: sempidan.tirtonadi.com: authenticated_id: rileyer+pandu.poluan.info/only user confirmed/virtual account not confirmed X-Source: X-Source-Args: X-Source-Dir: X-Archives-Salt: b43ad9fa-e703-4ea4-808a-05115aaf031e X-Archives-Hash: 319f078dd2321a4267474d6251c3ef39 --001a1135ecb86a03ed0506711cef Content-Type: text/plain; charset=UTF-8 On Oct 28, 2014 12:38 AM, "Rich Freeman" wrote: > > On Mon, Oct 27, 2014 at 1:23 PM, Volker Armin Hemmann > wrote: > > Am 27.10.2014 um 16:36 schrieb Rich Freeman: > >> and a boot > >> partition as I don't think grub supports it - it could be a bit of a > >> PITA for a single-drive system. > > > > nope. But I don't see any reason to use zfs with a single drive either. > > True, not needing to use FUSE does simplify things, but I don't > believe that grub supports zfs, so you would need a boot partition. > Granted, a newer laptop would need that for EFI anyway. > > > > >> However, it is probably more mature > >> than btrfs overall, and it certainly supports send. > > > > and if your send stream is corrupted, your data is gone. That is why I > > prefer cp&tar to backup my zfs data tank. > > > > If you ONLY save the send stream without checking it, then you're > right that you're depending on its integrity. I'd certainly be > nervous about doing that with btrfs, probably less so with zfs but I > can't really vouch for it. I don't know what ability either > filesystem gives you to verify a send stream in isolation. > > Now, what you could do is receive the send stream into a replica > filesystem on the far end, and not consider the backup successful > until this is done. That would look like a btrfs-to-btrfs rsync > operation, but it would be much more efficient in terms of IO. It > would require a daemon on the far end to run the receive operation and > report back status, vs just dumping the files via scp, etc. > > Does anybody know if either btrfs or zfs send includes checksums? I > know the data is checksummed on disk, but I have no idea if it is > protected in this way while serialized. > zfs has checksum for the send stream. That's why you can send the stream to a file, and fail to import the file sometime later if something changes in that file. So, always do a filesystem replication. Don't just save the send stream. Have the replica make the snapshots visible in poolroot/.zfs, and backup the whole filesystem using a deduping backup system. Rgds, -- --001a1135ecb86a03ed0506711cef Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Oct 28, 2014 12:38 AM, "Rich Freeman" <rich0@gentoo.org> wrote:
>
> On Mon, Oct 27, 2014 at 1:23 PM, Volker Armin Hemmann
> <volkerarmin@googlema= il.com> wrote:
> > Am 27.10.2014 um 16:36 schrieb Rich Freeman:
> >>=C2=A0 and a boot
> >> partition as I don't think grub supports it - it could be= a bit of a
> >> PITA for a single-drive system.
> >
> > nope. But I don't see any reason to use zfs with a single dri= ve either.
>
> True, not needing to use FUSE does simplify things, but I don't > believe that grub supports zfs, so you would need a boot partition. > Granted, a newer laptop would need that for EFI anyway.
>
> >
> >>=C2=A0 However, it is probably more mature
> >> than btrfs overall, and it certainly supports send.
> >
> > and if your send stream is corrupted, your data is gone. That is = why I
> > prefer cp&tar to backup my zfs data tank.
> >
>
> If you ONLY save the send stream without checking it, then you're<= br> > right that you're depending on its integrity.=C2=A0 I'd certai= nly be
> nervous about doing that with btrfs, probably less so with zfs but I > can't really vouch for it.=C2=A0 I don't know what ability eit= her
> filesystem gives you to verify a send stream in isolation.
>
> Now, what you could do is receive the send stream into a replica
> filesystem on the far end, and not consider the backup successful
> until this is done.=C2=A0 That would look like a btrfs-to-btrfs rsync<= br> > operation, but it would be much more efficient in terms of IO.=C2=A0 I= t
> would require a daemon on the far end to run the receive operation and=
> report back status, vs just dumping the files via scp, etc.
>
> Does anybody know if either btrfs or zfs send includes checksums?=C2= =A0 I
> know the data is checksummed on disk, but I have no idea if it is
> protected in this way while serialized.
>

zfs has checksum for the send stream. That's why you can= send the stream to a file, and fail to import the file sometime later if s= omething changes in that file.

So, always do a filesystem replication. Don't just save = the send stream. Have the replica make the snapshots visible in poolroot/.z= fs, and backup the whole filesystem using a deduping backup system.

Rgds,
--

--001a1135ecb86a03ed0506711cef--