Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wol <antlists@youngman.org.uk>]

On 03/02/2024 16:02, Grant Edwards wrote:
> rsnapshot is an application that uses rsync to do
> hourly/daily/weekly/monthly (user-configurable) backups of selected
> directory trees. It's done using rsync to create snapshots. They are
> in-effect "incremental" backups, because the snapshots themselves are
> effectively "copy-on-write" via clever use of hard-links by rsync. A
> year's worth of backups for me is 7 daily + 4 weekly + 12 monthly
> snapshots for a total of 23 snapshots.  If nothing has changed during
> the year, those 23 snapshots take up the same amount of space as a
> single snapshot.

So as I understand it, it looks like you first do a "cp with hardlinks" 
creating a complete new directory structure, but all the files are 
hardlinks so you're not using THAT MUCH space for your new image?

Then rsync copies and replaces any files that have been modified?

So each snapshot is using the space required by the directory structure, 
plus the space required by any changed files.
> 
> My understanding of ZFS is that it has built-in snapshot functionality
> that provides something similar to what is done by rsync by its use of
> hard-links.  In my current setup, there's an application called
> rsnapshot that manages/controls the snapshots by invoking rsync in
> various ways. My question was about the existence of a similar
> application that can be used with ZFS's built-in snapshot support to
> provide a similar backup scheme.

ZFS is a "copy on write" filesystem, I believe. So any changed blocks 
are rewritten to new blocks, the live snapshot is updated, and if the 
original block is required by older snapshots it is retained, else it is 
freed. And then you can push a snapshot to another disk. I think ZFS 
also has "dedupe on write", so if you're regularly pushing snapshots 
you're not wasting space with duplicate data.

And that is why I like "ext over lvm copying with rsync" as my strategy 
(not that I actually do it). You have lvm on your backup disk. When you 
do a backup you do "rsync with overwrite in place", which means rsync 
only writes blocks which have changed. You then take an lvm snapshot 
which uses almost no space whatsoever.

So to compare "lvm plus overwrite in place" to "rsnapshot", my strategy 
uses the space for an lvm header and a copy of all blocks that have changed.

Your strategy takes a copy of the entire directory structure, plus a 
complete copy of every file that has changed. That's a LOT more.

If my hard disk changes by lets say 0.1% a day, and I take daily 
snapshots, that's three years before I need to start deleting backups 
assuming I'm actually using half my disk (and with terabyte disks, both 
the amount of change, and the amount of disk used, is likely to be a lot 
less than those figures).

Cheers,
Wol