From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id D09D315815E for ; Sun, 4 Feb 2024 06:24:19 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 6422BE2B0B; Sun, 4 Feb 2024 06:24:13 +0000 (UTC) Received: from ciao.gmane.io (ciao.gmane.io [116.202.254.214]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id E77D1E2B02 for ; Sun, 4 Feb 2024 06:24:12 +0000 (UTC) Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1rWVvF-0008NO-VU for gentoo-user@lists.gentoo.org; Sun, 04 Feb 2024 07:24:09 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-user@lists.gentoo.org From: Grant Edwards Subject: [gentoo-user] Re: Suggestions for backup scheme? Date: Sun, 4 Feb 2024 06:24:05 -0000 (UTC) Message-ID: References: <8f5371a5-07af-456e-8517-cb9bb664fac4@youngman.org.uk> User-Agent: slrn/1.0.3 (Linux) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-Archives-Salt: dea6915c-9f2f-4897-8d88-7d158be9f9b1 X-Archives-Hash: 8aeefd86d77e59996e0b1508b4fe1bda On 2024-02-03, Wol wrote: > On 03/02/2024 16:02, Grant Edwards wrote: >> rsnapshot is an application that uses rsync to do >> hourly/daily/weekly/monthly (user-configurable) backups of selected >> directory trees. It's done using rsync to create snapshots. They are >> in-effect "incremental" backups, because the snapshots themselves are >> effectively "copy-on-write" via clever use of hard-links by rsync. A >> year's worth of backups for me is 7 daily + 4 weekly + 12 monthly >> snapshots for a total of 23 snapshots. If nothing has changed during >> the year, those 23 snapshots take up the same amount of space as a >> single snapshot. > > So as I understand it, it looks like you first do a "cp with hardlinks" > creating a complete new directory structure, but all the files are > hardlinks so you're not using THAT MUCH space for your new image? No, the first snaphost is a complete copy of all files. The snapshots are on a different disk, in a different filesystem, and they're just plain directory trees that you can brose with normal filesystem tools. It's not possible to hard-link between the "live" filesystem and the backup snapshots. The hard-links are to inodes "shared" between different snapshot directory trees. The first snapshot copies everything to the backup drive (using rsync). The next snapshot creates a second directory tree with all unchanged files hard-linked to the files that were copied as part of the first snapshot. Any changed files just-plain-copied into the second snapshot directory tree. The third snapshot does the same thing (starting with the second snapshot directory tree). Rinse and repeat. Old snapshots trees are simply removed a-la 'rm -rf" when they're no longer wanted. > So each snapshot is using the space required by the directory > structure, plus the space required by any changed files. Sort of. The backup filesystem has to contain one copy of every file so that there's something to hard-link to. The backup is completely stand-alone, so it doesn't make sense to talk about all of the snapshots containing only deltas. When you get to the "oldest" snapshot, there's nothing to delta "from". > [...] > > And that is why I like "ext over lvm copying with rsync" as my > strategy (not that I actually do it). You have lvm on your backup > disk. When you do a backup you do "rsync with overwrite in place", > which means rsync only writes blocks which have changed. You then > take an lvm snapshot which uses almost no space whatsoever. > > So to compare "lvm plus overwrite in place" to "rsnapshot", my > strategy uses the space for an lvm header and a copy of all blocks > that have changed. > > Your strategy takes a copy of the entire directory structure, plus a > complete copy of every file that has changed. That's a LOT more. I don't understand, are you saying that somehow your backup doesn't contain a copy of every file? -- Grant