From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1MXE2m-0007K7-N5 for garchives@archives.gentoo.org; Sat, 01 Aug 2009 12:50:41 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id E58C6E04C6; Sat, 1 Aug 2009 12:50:34 +0000 (UTC) Received: from mail-fx0-f211.google.com (mail-fx0-f211.google.com [209.85.220.211]) by pigeon.gentoo.org (Postfix) with ESMTP id 8C420E04C6 for ; Sat, 1 Aug 2009 12:50:34 +0000 (UTC) Received: by fxm7 with SMTP id 7so1762760fxm.34 for ; Sat, 01 Aug 2009 05:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:subject :message-id:in-reply-to:references:x-mailer:mime-version :content-type; bh=pEc6w6V8FzyUjfJvoN96VekN57pgRZE59BVIDYZVOu8=; b=lMODPcaOuciJg7KEmlwZwNY1Z3HpdEyEkYIVMfzwEXLknAj0h851JzUpX/UTtSnGDd uUVi4phwSlCi5WGTe0kZZGE97YKNNCvDB6FvOTka5/6edd7Ok2rlMtL3HuRz/A7kznxd qhtH40iybc7xZc46XpOzmJQZB7UcYjSPdf3M0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=rUsOCcLQ6JlDAGjqw+CX8AijUVpISkMcPD5uIrvdGY7FI2OO3Hao07TSU/oPAZOJ1v 0GOddobQHRFQpGQSCtWaCylfNLjiey8v5SM9G+j1khgiZqCAgOPBu2IsLlBr6p5eZbGj 32gxs+6xocovN+pGbRJfPSuLIVitpx1j0Ms5I= Received: by 10.204.52.146 with SMTP id i18mr924255bkg.82.1249131032496; Sat, 01 Aug 2009 05:50:32 -0700 (PDT) Received: from sacrilege ([91.191.238.58]) by mx.google.com with ESMTPS id 9sm5042315fks.28.2009.08.01.05.50.30 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 01 Aug 2009 05:50:31 -0700 (PDT) Date: Sat, 1 Aug 2009 18:50:21 +0600 From: Mike Kazantsev To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] File synchronisation utility (searching for/about to program it) Message-ID: <20090801185021.5253f31a@sacrilege> In-Reply-To: <5f14cf5e0907251010h51b7b538o79951d74b7470c5d@mail.gmail.com> References: <5f14cf5e0907220809ud14a99dq81950fba1c45b495@mail.gmail.com> <7bef1f890907241943k7c6a6fckb79a90816cae280f@mail.gmail.com> <5f14cf5e0907251010h51b7b538o79951d74b7470c5d@mail.gmail.com> X-Mailer: Claws Mail 3.7.2 (GTK+ 2.16.5; x86_64-pc-linux-gnu) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/1OfuzFaiUMM8SY7QyrvY0Rz"; protocol="application/pgp-signature" X-Archives-Salt: ee14fad5-d0e6-4578-9e57-3ac92d461d0b X-Archives-Hash: c5f3194f008b5af5742fb75e2b53906a --Sig_/1OfuzFaiUMM8SY7QyrvY0Rz Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sat, 25 Jul 2009 13:10:41 -0400 Simon wrote: > I have tried using git in the past and found that it doesnt work in my > 'space constrained' scenario. The need for a repository is a problem. > The use of the usbkey however is nice since it allows git to work > without having each computer maintain its own repository... but > still... i dont currently have a usbkey that's large enough to hold > all my data, even if i could compress it i doubt it would fit. >=20 > Another thing is, i wonder if it retains the attributes of the file > (creation date, mod date, owner/group, permissions)? As this can be > important on some aspects of my synchronisation needs. Vanilla git doesn't, apart from executable bit. Due to highly-modular structure of git, one can easily implement it as a wrapper or replacement binary at some level, storing metadata in some form (plain list, mirror tree or just alongside each file) when pushing changes to repo, applying on each pull. Then there are also git-hooks, which should be a better way than wrapper in theory, but I found them much harder to use in practice. > Still, git is a very good solution that works incrementally in a > differential manner (makes patches from previous versions). But when > i tried it, i found to suit my needs it would require the programming > of a big wrapper that would interface git to make some daily quick > actions simpler than a few git commands. That's another advantage of wrapper, but note that git-commands themselves can be quite extensible via aliases, configurable in gitconfig at any level (repo, home, system-wide). [alias] ci =3D commit -a co =3D checkout st =3D status -a br =3D branch ru =3D remote update ui =3D update-index --refresh cp =3D cherry-pick Still, things such are "git ui && git cp X" are quite common, so wrapper, or at least a set of shell aliases is quite handy. >> I apologize if the existence of a bare repo as an intermediary is a prob= lem. >> This can be done on a server as well. =20 > > It is... it makes all my computer dependant on that repo... sync'ing > computers at home can be done alright, but will still require walking > around pluging/unpluging. Makes this practically impossible to do > over the network (or to sync my host on the internet, not all my pc > are connected to the internet so the repo cant be just on the server, > i would have to maintain several repositories to work this out...). > It may be possible to adapt it to my scenario, but i think it will > require a lot of design in advance... but i'll check it out... at > worst it will convince me i should program my own, better it will give > me some good ideas or fortify some of my own good ideas and at best it > will be the thing i've been looking for! Why keep bare repo at all? That's certainly not a prequisite with distributed VCS like git. You can fetch / merge / rebase / cherry-pick commits with git via ssh just as easy as with rsync, using some intermediate media only if machines aren't connected at all, but then there's just no way around it. And even here, knowing approximate date of last sync, you can use commands like git-bundle to create single pack of new objects, which remote(s) can easily import, transferring this via any applicable method or protocol between / to any number of hosts. As you've noted already, git is quite efficient when it comes to storage, keeping the changes alone. When this will become a problem due to long history of long-obsoleted changes, you can drop them all, effectively 'sqashing' all the commits in one of the repos, rebasing the rest against it. So that should cover requirement one. Cherry-picking commits or checking out individual files / dirs on top of any base from any other repo/revision is pretty much what is stated in the next three requirements. One gotcha here is that you should be used to making individual commits consistent and atomic, so each set of changes serves one purpose and you won't be in situation when you'll need "half of commit" anywhere. Conflict resolution is what you get with merge / rebase (just look at the fine "git-merge" man page), but due to abscence of "ultimate AI" these better used repeatedly against the same tree. About the last point of original post... I don't think git is "intuitive" until you understand exactly how it works - that's when it becomes one, with all the high-level and intermediate interfaces having great manpage and sole, clear purpose. That said, I don't think git is the best way to sync everything. I don't mix binary files with configuration, because just latter suffice with gentoo: you have git-synced portage (emerge can sync via VCS out-of-the-box), git-controlled overlay on top of it and pull the world/sets/flag/etc changes... just run emerge and you're set, without having to worry about architectural incompatibilities of binaries or missing misc libs, against which they're linked here and there. That's what portage is made for, after all. Just think of trendemous space efficiency here - no binaries are backed up at all, and all you need to do to restore 3G root from 2M pack is "git clone (or receive-pack) && emerge -uDN @world" ;) --=20 Mike Kazantsev // fraggod.net --=20 Mike Kazantsev // fraggod.net --Sig_/1OfuzFaiUMM8SY7QyrvY0Rz Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) iEYEARECAAYFAkp0OhMACgkQASbOZpzyXnHXBQCfe6kzdmtw4dH+MOYChPPBnlwI htcAnR/xtsYph5uDhbSf72g7d9us8a/A =flHb -----END PGP SIGNATURE----- --Sig_/1OfuzFaiUMM8SY7QyrvY0Rz--