From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1O9oOq-0008Tb-Iy for garchives@archives.gentoo.org; Wed, 05 May 2010 23:53:08 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 16003E065E; Wed, 5 May 2010 23:52:36 +0000 (UTC) Received: from mail2.pcorp.com.au (mail2.pcorp.com.au [150.101.72.19]) by pigeon.gentoo.org (Postfix) with ESMTP id BB09BE065E for ; Wed, 5 May 2010 23:52:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail2.pcorp.com.au (Postfix) with ESMTP id A080AA00015 for ; Thu, 6 May 2010 09:22:31 +0930 (CST) X-Virus-Scanned: amavisd-new at mail2.pcorp.com.au Received: from mail2.pcorp.com.au ([127.0.0.1]) by localhost (mail2.pcorp.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 001FYGMo38Uf; Thu, 6 May 2010 09:22:30 +0930 (CST) Received: from [172.16.0.52] (unknown [172.16.0.52]) by mail2.pcorp.com.au (Postfix) with ESMTP id BB416A00013 for ; Thu, 6 May 2010 09:22:30 +0930 (CST) Subject: Re: [gentoo-user] kernel notification of file system changes From: Iain Buchanan To: gentoo-user@lists.gentoo.org In-Reply-To: References: <1273042474.20354.17.camel@localhost> <07250F7A-39A9-4417-A0E8-CBCD4E8CDDC6@stellar.eclipse.co.uk> <4BE1A9C5.5090709@f_philipp.fastmail.net> Content-Type: text/plain; charset="ISO-8859-1" Date: Thu, 06 May 2010 09:21:08 +0930 Message-ID: <1273103468.20354.45.camel@localhost> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org Mime-Version: 1.0 X-Mailer: Evolution 2.28.3.1 Content-Transfer-Encoding: 7bit X-Archives-Salt: 01d70d6d-562b-4a6b-9694-7455b8c46c4b X-Archives-Hash: a53f5786cfe6cd8053ee0598d58bc188 On Wed, 2010-05-05 at 18:35 +0100, Stroller wrote: > On 5 May 2010, at 18:24, Florian Philipp wrote: > >> ... > >> man inotify(7): > >> ... When a directory is monitored, inotify will return events for the > >> directory itself, and for files inside the directory. > >> ... > > > > To repeat my comment on Iain's original "backup to a cold-swap drive" > > thread ... > > Sorry, I started ignoring that almost immediately it was posted. ooohh, ouch! :) > He > rejected too quickly too many workable solutions to basically > functional backup. Perhaps Iain is a perfectionist, but I did not wish > to follow the thread. Perhaps I am a bit of a perfectionist, but I think you misunderstood my aim. I rejected 3 options straight away (dd, gparted, and Ghost4Linux) because they're not designed for backing up a live filesystem in a "change only" fashion ("intelligent" is the word I used). Beyond that I didn't reject anything else that anyone mentioned. > > ... Inotify has two drawbacks which make it hard or even impossible > > to use for Iain's use case: > > > > a) It does not work recursively which means that you have to create a > > new handle for each subdirectory. Of course, this only means more work > > for the programmer but there is also the problem that > > Pardon me. I assumed that "files inside the directory" meant that foo > would be be changed when foo/bar changed, thus monitoring grunt would > reflect changes in grunt/foo/bar. I overlooked that a directory is not > a file. isn't it? I thought a directory was just a file containing names (or inodes) of other files? Which would explain why monitoring grunt wouldn't show changes in grunt/foo/bar, since the directory/file called grunt remains the same (ie. contains the same list of inodes) even if grunt/foo/bar changes. Let me tell you what I actually want to do, which I may not have made clear originally: I want to backup root to an external drive (or that could be rephrased as "I want to backup any mount to any other mount), such that: 1. My backup is an hour or so out of date (at most) 2. I don't need to copy the entire filesystem every time To do that, I could either: * Run rsync every hour over the entire filesystem (I'm doing this now with ionice, takes about 10 minutes when there are no changes) * Use some file notification monitor to tell me which file was just changed, and only rsync that file The problems with rsync is that during the rsync process, the filesystem is changing, so I will end up with a slightly inconsistent backup. If I use some notification method that tells me a file has changed, I can greatly reduce any inconsistency, and I reduce my hour down to seconds or minutes, depending on how much changes at any one time. I'm considering LVM for it's snapshot capability, but I'd still have to rsync root. I would prefer a file notification method as well, so I can just rsync the file that just changed. So far all the file monitoring tools are based on individual files (even the recursive ones), and you eventually reach a system limit. Thanks, and willing to listen to any ideas from anyone (except Stroller :p ) -- Iain Buchanan Usually, when a lot of men get together, it's called a war. -- Mel Brooks, "The Listener"