From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id A9C47139337 for ; Sun, 1 Aug 2021 03:37:34 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 48C39E0995; Sun, 1 Aug 2021 03:37:27 +0000 (UTC) Received: from icp-osb-irony-out5.external.iinet.net.au (icp-osb-irony-out5.external.iinet.net.au [203.59.1.221]) by pigeon.gentoo.org (Postfix) with ESMTP id F1FE6E0977 for ; Sun, 1 Aug 2021 03:37:24 +0000 (UTC) X-SMTP-MATCH: 0 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AD4kvCK8q5GsSVvD9k2Buk+DfI+orL9Y04l?= =?us-ascii?q?Q7vn2ZOiYlCvBw8Pre5cjztCWE7gr5N0tQ++xoVJPhfZqYz+8T3WBzB8bHYO?= =?us-ascii?q?CFghrKEGgK1+KLqFCNJ8S9zI9gPMxbHJSWZuefMbE3t7ee3OBmKbwd/OU=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2DJBgCQFgZh/+hY69xaHAEBAQEBAQc?= =?us-ascii?q?BARIBAQQEAQFACYFQgXyBfAEBAYUwiQSHEQEBAQEBAQaBFC0DOAGECIR/gSm?= =?us-ascii?q?SdAsBAQEBAQEBAQEJQQQBAYRYAoMAJjgTAQIEFQEBAQUBAQEBAQYDAYEkhXW?= =?us-ascii?q?GQwEFIw8BIxEiCxgCAiYCAlcTCAEBgm2CYiWrD4ExgQGEaIUdgRAqjW9DfYE?= =?us-ascii?q?QgTwPgm8+g34NG2uCSoJkBIMQBmItASeBFHZCjEqEdxOOUosFkhCDMZdohks?= =?us-ascii?q?GDwUmlU4IkRGGd68cDzaEcoE+OYF9TR8ZgyVPGQ5XjVQWFW4BCI0uNAEBAWc?= =?us-ascii?q?CBgEKAQEDCYgJgWgBXgEB?= X-IPAS-Result: =?us-ascii?q?A2DJBgCQFgZh/+hY69xaHAEBAQEBAQcBARIBAQQEAQFAC?= =?us-ascii?q?YFQgXyBfAEBAYUwiQSHEQEBAQEBAQaBFC0DOAGECIR/gSmSdAsBAQEBAQEBA?= =?us-ascii?q?QEJQQQBAYRYAoMAJjgTAQIEFQEBAQUBAQEBAQYDAYEkhXWGQwEFIw8BIxEiC?= =?us-ascii?q?xgCAiYCAlcTCAEBgm2CYiWrD4ExgQGEaIUdgRAqjW9DfYEQgTwPgm8+g34NG?= =?us-ascii?q?2uCSoJkBIMQBmItASeBFHZCjEqEdxOOUosFkhCDMZdohksGDwUmlU4IkRGGd?= =?us-ascii?q?68cDzaEcoE+OYF9TR8ZgyVPGQ5XjVQWFW4BCI0uNAEBAWcCBgEKAQEDCYgJg?= =?us-ascii?q?WgBXgEB?= X-IronPort-AV: E=Sophos;i="5.84,285,1620662400"; d="scan'208";a="390875577" Received: from 220-235-88-232.dyn.iinet.net.au (HELO mail.infra.localdomain) ([220.235.88.232]) by icp-osb-irony-out5.iinet.net.au with ESMTP; 01 Aug 2021 11:37:22 +0800 Received: from localhost (mail.infra.localdomain [127.0.0.1]) by mail.infra.localdomain (Postfix) with ESMTP id 0505F41E for ; Sun, 1 Aug 2021 11:37:22 +0800 (AWST) X-Virus-Scanned: amavisd-new at localdomain Received: from mail.infra.localdomain ([127.0.0.1]) by localhost (mail.infra.localdomain [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lVIKXdCv8_re for ; Sun, 1 Aug 2021 11:36:37 +0800 (AWST) Subject: Re: [gentoo-user] [OT] SMR drives (WAS: cryptsetup close and device in use when it is not) To: gentoo-user@lists.gentoo.org References: <96fc901a-2ce4-0ea0-0ed1-1c529145c0e9@gmail.com> <6102DB58.7040103@youngman.org.uk> <56d64f52-1b9a-1309-c720-06bb63c9f80a@iinet.net.au> <7a8c52c3-4c96-89ac-ace0-6eb4b8f1401f@iinet.net.au> <6104C897.5010505@youngman.org.uk> <7e6b56a2-917f-4dd6-3599-722bca5162ca@iinet.net.au> From: William Kenworthy Message-ID: Date: Sun, 1 Aug 2021 11:36:36 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-AU X-Archives-Salt: 80cbb027-58dd-47fb-8e96-e5110a6ed779 X-Archives-Hash: afe42aa1e7730247efc854c214c9cc3c On 1/8/21 8:50 am, Frank Steinmetzger wrote: > Am Sat, Jul 31, 2021 at 12:58:29PM +0800 schrieb William Kenworthy: > >> Its not raid, just a btrfs single on disk (no partition).  Contains a >> single borgbackup repo for an offline backup of all the online >> borgbackup repo's I have for a 3 times a day backup rota of individual >> machines/data stores > So you are borg’ing a repo into a repo? I am planning on simply rsync’ing > the borg directory from one external HDD to another. Hopefully SMR can cope > with this adequatly. > > And you are storing several machines into a single repo? The docs say this > is not supported officially. But I have one repo each for /, /home and data > for both my PC and laptop. Using a wrapper script, I create snapshots that > are named $HOSTNAME_$DATE in each repo. Basicly yes: I use a once per hour snapshot of approximately 500Gib of data on moosefs, plus borgbackups 3 times a day to individual repos on moosefs for each host.  3 times a day, the latest snapshot is stuffed into a borg repo on moosefs and the old  snapshots are deleted.  I currently manually push all the repos into a borg repo on the USB3 SMR drive once a day or so. 1. rsync (and cp etc.) are dismally slow on SMR - use where you have to, avoid otherwise. 2. borgbackup with small updates goes very quick 3. borgbackup often to keep changes between updates small - time to backup will stay short. 4. borg'ing a repo into a repo works extreemly well - however there are catches based around backup set names and the file change tests used. (ping me if you want the details) 5. Yes, I have had disasters (i.e., a poorly thought out rm -rf in a moosefs directory, unstable power that took awhile to cure, ...) requiring underfire restoration of both large and small datasets - it works! 6. Be careful of snapshot resources on moosefs - moosefs has a defined amount of memory for each file stored.  Even with the lazy snapshot method, taking a snapshot will about double the memory usage on the master for that portion of the filesystem.  Also taking too many snapshots multiplies the effect.  Once you go into swap, it becomes a recovery effort.  Also keep in mind that trashtime is carried into the snapshot so the data may exist in trash even after deletion - its actually easy to create a DOS condition by not paying attention to this. BillK > >> - I get an insane amount of de-duplication that way for a slight decrease >> in conveniance! > And thanks to the cache, a new snapshots usually is done very fast. But for > a yet unknown reason, sometimes Borg re-hashes all files, even though I > didn’t touch the cache. In that case it takes 2½ hours to go through my > video directory. >