From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id A3B1215815E for ; Wed, 7 Feb 2024 22:36:51 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 1470BE2A47; Wed, 7 Feb 2024 22:36:46 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 328C3E2A34 for ; Wed, 7 Feb 2024 22:36:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1707345398; x=1707950198; i=warp_7@gmx.de; bh=wjurTXVgC+my29d9RqRZQgdvRg914eLZLBRbrebVD1Q=; h=X-UI-Sender-Class:Date:From:To:Subject:References:In-Reply-To; b=sBpDH6cG1N9G3Bd9ycv9N5lGUhdIlBxqt9GhwGQYZz6uq3Jae+ByqYHTnWBg+72r eiklMYbj/SLxSrBY57+89dgEcY3OPNfW7y59tR/n6cSC7z09m7tbsNsOyaFfxvmbL OQqvuE/tyACbD5IxwluGD5XJ1vXZszv0XumgBz/HtOOUTS0z6FLBkBcXtTem5/JsW Id5oybdthysWSMMTkxNYHYx1/i1M9uqSQP4TqgKCYMBQMW4EhkQJALG9ZjsejvxVc Q6K/e6ro2floL/bfngftjhT8A6l2qfhIK6AH4IatGfHotel6NIY10Gda1HNlJOk9q 4L/5JoDItjmMBAr3cA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from schatulle ([89.245.63.38]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MSbxD-1rRdF91emn-00T08V for ; Wed, 07 Feb 2024 23:36:38 +0100 Date: Wed, 7 Feb 2024 23:36:36 +0100 From: Frank Steinmetzger To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Suggestions for backup scheme? Message-ID: Mail-Followup-To: gentoo-user@lists.gentoo.org References: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="XYlM/0ks8JAMn1+u" Content-Disposition: inline In-Reply-To: User-Agent: Mutt 2.2.12; VIM - Vi IMproved 9.1 X-Provags-ID: V03:K1:QyVGJizNrOnVh+cRCIbbTOt5Udns2qnD/61ieOPzIoxC/YGE+oa hc/xnxtF/FddM8kkUuYVd/PAt2rZGa+jyvOEOV8iX2NAk5rzWjLPl6/jiTlC+d1x4PhUVQT dPlj/aa806sShbSGml041J894AitPLUM6trzm50WCkfeJdFXIEjHeJNiD1SazqAkSmbO5gq 7SMGE1R9+PBTTnJPT+6hg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:hKTOQ8lktO8=;nLaPVO334ANUvfH3u3OAEmOIV+X YqcdhdpHZe2jH3Sl1Fpo5UkE33mznlnb1A5RSFuM44KlconcXY5uHNiLD5GRxW0AT7lcddU8c Alr3ogB9A9tT6vAdlA9VSPqLytnZ+S2A27ydafxi6QIKfw4/WkdxnIHOSwQSz0UPVkD/MShF5 +s4tjAaRVzDDIfvF941rMzoopVtlcor6xt9qukZ6FXufq/5d6/IE67pwcvEtG79OfS1JBpTjS t43Nvtu86fJrE3L0McJzFoOYjAZyOGWH3N1xaLYr/NdE96Y6lAnApGw2r9TWZ/thzxrQfW+Px mlWkaHHMpclsT2N1bMF3GRccZS+5qt9pBM3Yf5OEyivsY2Us/gv9k1uqWO1SPJNW+kFgg6zGa QEDQjGq73Gel+TpZmvx2JUrKltBiQH8CAHLEhZLOPDetXjV68U9Egjm1Mu4Xkf34v5B9V/Fkn A82qJ4z10WmdfsL1wi3QefgYbx+ct7G8zhV3sYKGhnPS9Cyy0ay/fz7a1Aj4pVFdAJtKTv+4c 0vdlbdwhciLe13K7wgi192rkaM7ESOSgHv0/Dqyeooe85eulcGG9ActKiVhxvKjJ8F4QDaIb5 a0hGxSeuvjXhvBNBGCCi2LNQdlKjS6CHXJtNX6KOWQWDdIC4T7qZL85WNwqIWT9Qqz1gpw8Hz dWiLwokzXWgONUNm32ofaaIfAt22LLDq5cRolnouFsFCr3uVgsHzvhnQ2F6mw8jqqLDJfbdg0 2TDQmw4kJySNMjpp/t8OPDO5n7SLeB5jjxaCwsMVZh3VYknUGIA4+VR60E2dPFMubKtRoP9wT w5tT9NUyj7oriZx4Q/uoHSlls4eLiXvH9C11/8sHYFf04= X-Archives-Salt: 683f8a0b-cd76-4240-978c-a2dc1cbdf096 X-Archives-Hash: 61a818f2c075dd94b0373d6de1bf2aca --XYlM/0ks8JAMn1+u Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Am Tue, Jan 30, 2024 at 06:15:09PM -0000 schrieb Grant Edwards: > I need to set up some sort of automated backup on a couple Gentoo > machines (typical desktop software development and home use). One of > them used rsnapshot in the past but the crontab entries that drove > that have vanished :/ (presumably during a reinstall or upgrade -- > IIRC, it took a fair bit of trial and error to get the crontab entries > figured out). >=20 > I believe rsnapshot ran nightly and kept daily snapshots for a week, > weekly snapshots for a month, and monthly snapshots for a couple > years. >=20 > Are there other backup solutions that people would like to suggest I > look at to replace rsnapshot? I was happy enough with rsnapshot (when > it was running), but perhaps there's something else I should consider? In my early backup times I, too, used rsnapshot to back up my ~ and rsync= =20 for my big media files. But that only included my PC. My laptop was wholly= =20 un-backed-up. I only syncronised much of my home and my audio collection=20 between the two with unison. At some point my external 3 TB drive became=20 free and then I started using borg to finally do proper backups. Borg is very similar to restic, I actually used the two in parallel for a= =20 while to compare them, but stayed with borg. One pain point was that I=20 couln=E2=80=99t switch off restic=E2=80=99s own password protection. Since = all my backup=20 disks are LUKSed anyway, I don=E2=80=99t need that. Since borg works block-based, it does deduplication without extra cost and= =20 it is suitable for big image files which don=E2=80=99t change much. I do fu= ll=20 filesystem backups of /, ~ and my media partition of my main PC and my=20 laptop. I have one repository for each of those three filesystems, and each= =20 repo receives the data from both machines, so they are deduped. Since both= =20 machines run Arch, their roots are binary identical. The same goes for my= =20 unison-synced homes. Borg has retention logic built-in. You can say I want to keep the latest=20 archive of each of the last 6 days/weeks/months/years, and it even goes dow= n=20 to seconds. And of course you can combine those rules. The only thing is=20 they don=E2=80=99t overlap, meaning if you want to keep the last 14 days an= d the=20 last four weeks, those weekly retentions start after the last daily=20 snapshots. In summary, advantages: + fast dedup, built-in compression (different algos and levels configurable) + big data files allow for quick mirroring of repositories. I simply rsync my primary backup disk to two other external HDDs. + Incremental backups are quite fast because borg uses a cache to detect changed files quickly. Disadvantages: - you need borg to mount the backups it - it is not as fast as native disk access, especially during restore and=20 when getting a total file listing due to lots of random I/O on the HDD. As example, I currently have 63 snapshots in my data partition repository: # borg list data/ tp_2021-06-07 Mon, 2021-06-07 16:27:44 [5f9ebd9f24353c340691b2a71= f5228985a41699d2e23473ae4e9e795669c8440] kern_2021-06-07 Mon, 2021-06-07 23:58:56 [19c76211a9c35432e6a66ac18= 92ee19a08368af28d2d621f509af3d45f203d43] [... 55 more lines ...] kern_2024-01-14 Sun, 2024-01-14 20:53:23 [499ce7629e64cffb7ec6ec9ff= bf0c595e4ede3d93f131a9a4b424b165647f645] tp_2024-01-14 Sun, 2024-01-14 20:57:42 [ea2baef3e4bb49c5aec7cf853= 6f7b00b55fb27ecae3a80ef9f5a5686a1da30d5] kern_2024-01-21 Sun, 2024-01-21 23:42:46 [71aa2ce6cf4021712f949af06= 8498bfda7797b5d1c5ddc0f0ce8862b89e48961] tp_2024-01-21 Sun, 2024-01-21 23:48:24 [45e35ed9206078667fa62d0e4= a1ac213e77f52415f196101d14ee21e79fc393d] kern_2024-02-04 Sun, 2024-02-04 23:16:43 [e1b015117143fad6b89cea663= 29faa888cffc990644e157b1d25846220c62448] tp_2024-02-04 Sun, 2024-02-04 23:23:15 [e9b167ceec1ab9a80cbdb1acf= 4ff31cd3935fc23e81674cad1b8694d98547aeb] The last =E2=80=9Ctp=E2=80=9D (Thinkpad) snapshot contains 1 TB, =E2=80=9Ck= ern=E2=80=9D (my PC) 809 GB. And here you see how much space this actually takes on disk: # borg info data/ [ ... ] Original size Compressed size Deduplicated size All archives: 56.16 TB 54.69 TB 1.35 TB Obviously, compression doesn=E2=80=99t do much for media files. But it is v= ery=20 effective in the repository for the root partitions: # borg info arch-root/ [ ... ] Original size Compressed size Deduplicated size All archives: 1.38 TB 577.58 GB 79.41 GB --=20 Gr=C3=BC=C3=9Fe | Greetings | Salut | Qapla=E2=80=99 Please do not share anything from, with or about me on any social network. =E2=80=9CShe understands. She doesn=E2=80=99t comprehend.=E2=80=9D =E2=80= =93 River Tam, Firefly --XYlM/0ks8JAMn1+u Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEVbE9o2D2lE5fhoVsizG+tUDUMMoFAmXEBfIACgkQizG+tUDU MMpeWg//e7zMTw2NuqH52LqKl08EFI4RIzZDuFAdqGkbGLXbZYnvOtNKILw/l1+g e8kQhqndhgky/1IczKKyA2nzEOQICr1tSFDcClOPGe4dH4qcdGOFjqjkqhhAtv8I Hy6pjAdvTjbOV/5VvD+yFJFBJPYl0lv3Yo7vXrFze6ZyfwjkJmPouIBm/PGH5tcV mhoQGrPo1gn4CuErQn1Bp2QEE6pQnZtfARjMUr+vuC+IsuVjWqKorjo0oR49mJY8 2LXd0LeZqMeoVz7WRfI9D0y5Gubl9VD2ICn+HzL4tQxTOEwMjO8pexgPdg5wZLAV ArZMWv3sOmeOQTtjl1jigbC8uqEAgVlTt/mJk9P2ZHlA94NURsvMThJKwHPSpSqd 8iINREnbMjqo3r8RmL39u6vy3/L7ag8XqGz5grYsfjKw+fX/rAJra5GjrhNsq32H YqmPKaaXt5rhrFkXZTOzjLzwTjLlBeoNsZIGIQijDrn+vb7DstnGo83ASutmkjnj BYTWmkiarQIRAemweNRNwG97uUolbAzMB608NYU8vXMMj5hlrolMtr7x89HS8HGA deeI7JMAE+0QKmgL5VtF8GIVp8ZdUOSPgekGDBIvodB4q98EYyJXfDamIE/GrQ65 95NpqgB+0xXx8TBDylhjBbWw2uk66pUL7HLvX8ppUeZn4LeNZWY= =8XYn -----END PGP SIGNATURE----- --XYlM/0ks8JAMn1+u--