public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-user] How to exclude a directory from rsync
@ 2010-11-14 21:03 Mick
  2010-11-14 22:47 ` Stefan G. Weichinger
  0 siblings, 1 reply; 16+ messages in thread
From: Mick @ 2010-11-14 21:03 UTC (permalink / raw
  To: gentoo-user

I am not sure how to exclude a directory on an ntfs partition from
being accessed during rsync.  The attributes do not seem to be right
and it comes up with this error:
===============================
'rsync -a -l -v --exclude "/mnt/User_WinXP/System Volume Information"
-e "ssh -c blowfish -l root" /mnt/User_WinXP/
10.10.10.25:/home/httpd/backup'

sending incremental file list
rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
Permission denied (13)

sent 975792 bytes  received 3760 bytes  23048.28 bytes/sec
total size is 11957666624  speedup is 12207.28
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1042) [sender=3.0.7]
===============================

Is my syntax wrong?

This what mount shows for the partition (I use ntfs-3g):

/dev/sda7 on /mnt/User_WinXP type fuseblk
(rw,nosuid,nodev,noatime,allow_other,blksize=4096)
-- 
Regards,
Mick



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-14 21:03 [gentoo-user] How to exclude a directory from rsync Mick
@ 2010-11-14 22:47 ` Stefan G. Weichinger
  2010-11-15 22:50   ` Mick
  0 siblings, 1 reply; 16+ messages in thread
From: Stefan G. Weichinger @ 2010-11-14 22:47 UTC (permalink / raw
  To: gentoo-user; +Cc: Mick

Am 14.11.2010 22:03, schrieb Mick:
> I am not sure how to exclude a directory on an ntfs partition from
> being accessed during rsync.  The attributes do not seem to be right
> and it comes up with this error:
> ===============================
> 'rsync -a -l -v --exclude "/mnt/User_WinXP/System Volume Information"

try something like "--exclude "./System Volume Information"

(relative paths)

or even "--exclude "./System\ Volume\ Information"

(escaping the spaces)

S




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-14 22:47 ` Stefan G. Weichinger
@ 2010-11-15 22:50   ` Mick
  2010-11-16  9:00     ` Stefan G. Weichinger
  0 siblings, 1 reply; 16+ messages in thread
From: Mick @ 2010-11-15 22:50 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: Text/Plain, Size: 833 bytes --]

On Sunday 14 November 2010 22:47:02 you wrote:
> Am 14.11.2010 22:03, schrieb Mick:
> > I am not sure how to exclude a directory on an ntfs partition from
> > being accessed during rsync.  The attributes do not seem to be right
> > and it comes up with this error:
> > ===============================
> > 'rsync -a -l -v --exclude "/mnt/User_WinXP/System Volume Information"
> 
> try something like "--exclude "./System Volume Information"
> 
> (relative paths)
> 
> or even "--exclude "./System\ Volume\ Information"
> 
> (escaping the spaces)

Thanks Stefan, I'm afraid I'm still getting the same problem:

rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
Permission denied (13)

Why is rsync trying to open this directory, when I thought I've asked it to 
exclude it?
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-15 22:50   ` Mick
@ 2010-11-16  9:00     ` Stefan G. Weichinger
  2010-11-16 13:28       ` Mick
  0 siblings, 1 reply; 16+ messages in thread
From: Stefan G. Weichinger @ 2010-11-16  9:00 UTC (permalink / raw
  To: gentoo-user; +Cc: Mick

Am 15.11.2010 23:50, schrieb Mick:

> Thanks Stefan, I'm afraid I'm still getting the same problem:
> 
> rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
> Permission denied (13)
> 
> Why is rsync trying to open this directory, when I thought I've asked it to 
> exclude it?

Maybe you did it wrong?

;-)

You don't show us what you did ...



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16  9:00     ` Stefan G. Weichinger
@ 2010-11-16 13:28       ` Mick
  2010-11-16 14:15         ` Alan McKinnon
                           ` (2 more replies)
  0 siblings, 3 replies; 16+ messages in thread
From: Mick @ 2010-11-16 13:28 UTC (permalink / raw
  To: gentoo-user

On 16 November 2010 09:00, Stefan G. Weichinger <lists@xunil.at> wrote:
> Am 15.11.2010 23:50, schrieb Mick:
>
>> Thanks Stefan, I'm afraid I'm still getting the same problem:
>>
>> rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
>> Permission denied (13)
>>
>> Why is rsync trying to open this directory, when I thought I've asked it to
>> exclude it?
>
> Maybe you did it wrong?
>
> ;-)
>
> You don't show us what you did ...

I ran the same as before but changed the path to the one you suggested:

===============================
'rsync -a -l -v --exclude "./System Volume Information"
-e "ssh -c blowfish -l root" /mnt/User_WinXP/
10.10.10.25:/home/httpd/backup'

sending incremental file list
rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
Permission denied (13)

[snip ...]

rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1042) [sender=3.0.7]
===============================

The System Volume is shown as 0700 (mounted with default permissions)
and is owned by root when viewed from Gentoo.
-- 
Regards,
Mick



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 13:28       ` Mick
@ 2010-11-16 14:15         ` Alan McKinnon
  2010-11-16 21:12           ` Mick
  2010-11-16 14:36         ` Stefan G. Weichinger
  2010-11-16 15:11         ` Robin Atwood
  2 siblings, 1 reply; 16+ messages in thread
From: Alan McKinnon @ 2010-11-16 14:15 UTC (permalink / raw
  To: gentoo-user

Apparently, though unproven, at 15:28 on Tuesday 16 November 2010, Mick did 
opine thusly:

> On 16 November 2010 09:00, Stefan G. Weichinger <lists@xunil.at> wrote:
> > Am 15.11.2010 23:50, schrieb Mick:
> >> Thanks Stefan, I'm afraid I'm still getting the same problem:
> >> 
> >> rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
> >> Permission denied (13)
> >> 
> >> Why is rsync trying to open this directory, when I thought I've asked it
> >> to exclude it?
> > 
> > Maybe you did it wrong?
> > 
> > ;-)
> > 
> > You don't show us what you did ...
> 
> I ran the same as before but changed the path to the one you suggested:
> 
> ===============================
> 'rsync -a -l -v --exclude "./System Volume Information"
> -e "ssh -c blowfish -l root" /mnt/User_WinXP/
> 10.10.10.25:/home/httpd/backup'
> 
> sending incremental file list
> rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
> Permission denied (13)
> 
> [snip ...]
> 
> rsync error: some files/attrs were not transferred (see previous
> errors) (code 23) at main.c(1042) [sender=3.0.7]
> ===============================
> 
> The System Volume is shown as 0700 (mounted with default permissions)
> and is owned by root when viewed from Gentoo.

Don't think of --exclude as being a file path match, think of it as more a 
regex (usually just a literal one). It specifies a pattern that if found if 
the full pathname, results in the file not being synced.

The string "./System Volume Information" of course appears nowhere in the 
rsync output list of files.

I think you want:

--exclude "/System Volume Information"

Then rsync will not attempt to open the directory at all

-- 
alan dot mckinnon at gmail dot com



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 13:28       ` Mick
  2010-11-16 14:15         ` Alan McKinnon
@ 2010-11-16 14:36         ` Stefan G. Weichinger
  2010-11-16 15:11         ` Robin Atwood
  2 siblings, 0 replies; 16+ messages in thread
From: Stefan G. Weichinger @ 2010-11-16 14:36 UTC (permalink / raw
  To: gentoo-user; +Cc: Mick

Am 16.11.2010 14:28, schrieb Mick:

> I ran the same as before but changed the path to the one you suggested:
> 
> ===============================
> 'rsync -a -l -v --exclude "./System Volume Information"
> -e "ssh -c blowfish -l root" /mnt/User_WinXP/
> 10.10.10.25:/home/httpd/backup'

Try it with escaped spaces as I mentioned also:

--exclude "./System\ Volume\ Information"

or even

--exclude "./System*"

if there is nothing else starting with "System" ;-)

S



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 13:28       ` Mick
  2010-11-16 14:15         ` Alan McKinnon
  2010-11-16 14:36         ` Stefan G. Weichinger
@ 2010-11-16 15:11         ` Robin Atwood
  2 siblings, 0 replies; 16+ messages in thread
From: Robin Atwood @ 2010-11-16 15:11 UTC (permalink / raw
  To: gentoo-user

On Tuesday 16 November 2010, Mick wrote:

Try:

'rsync -a -l -v --exclude "*/System Volume Information"
-e "ssh -c blowfish -l root" /mnt/User_WinXP/
10.10.10.25:/home/httpd/backup'

From "man rsync"
"Note that if you don’t specify --backup-dir, (1) the --omit-dir-times option 
will be implied, and (2) if --delete is also in effect (without --delete-
excluded), rsync will add a "protect" filter-rule for the backup suffix to the 
end of all your existing excludes (e.g. -f "P *~"). This will prevent 
previously backed-up files from being deleted. Note that if you are supplying 
your own filter rules, you may need to manually insert your own 
exclude/protect rule somewhere higher up in the list so that it has a high 
enough priority to be effective (e.g., if your rules specify a trailing 
inclusion/exclusion of ’*’, the auto-added rule would never be reached)."

HTH
-Robin
-- 
----------------------------------------------------------------------
Robin Atwood.

"Ship me somewheres east of Suez, where the best is like the worst,
 Where there ain't no Ten Commandments an' a man can raise a thirst"
         from "Mandalay" by Rudyard Kipling
----------------------------------------------------------------------











^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 14:15         ` Alan McKinnon
@ 2010-11-16 21:12           ` Mick
  2010-11-16 21:24             ` Alan McKinnon
  0 siblings, 1 reply; 16+ messages in thread
From: Mick @ 2010-11-16 21:12 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: Text/Plain, Size: 1606 bytes --]

On Tuesday 16 November 2010 14:15:00 Alan McKinnon wrote:
> Apparently, though unproven, at 15:28 on Tuesday 16 November 2010, Mick did
> opine thusly:
> > On 16 November 2010 09:00, Stefan G. Weichinger <lists@xunil.at> wrote:
> > > Am 15.11.2010 23:50, schrieb Mick:

> > > You don't show us what you did ...
> > 
> > I ran the same as before but changed the path to the one you suggested:
> > 
> > ===============================
> > 'rsync -a -l -v --exclude "./System Volume Information"
> > -e "ssh -c blowfish -l root" /mnt/User_WinXP/
> > 10.10.10.25:/home/httpd/backup'
> > 
> > sending incremental file list
> > rsync: opendir "/mnt/User_WinXP/System Volume Information" failed:
> > Permission denied (13)
> > 
> > [snip ...]
> > 
> > rsync error: some files/attrs were not transferred (see previous
> > errors) (code 23) at main.c(1042) [sender=3.0.7]
> > ===============================
> > 
> > The System Volume is shown as 0700 (mounted with default permissions)
> > and is owned by root when viewed from Gentoo.
> 
> Don't think of --exclude as being a file path match, think of it as more a
> regex (usually just a literal one). It specifies a pattern that if found if
> the full pathname, results in the file not being synced.
> 
> The string "./System Volume Information" of course appears nowhere in the
> rsync output list of files.
> 
> I think you want:
> 
> --exclude "/System Volume Information"
> 
> Then rsync will not attempt to open the directory at all

Excellent, it worked!  :-)

Thank you all for your help.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 21:12           ` Mick
@ 2010-11-16 21:24             ` Alan McKinnon
  2010-11-16 22:26               ` Stefan G. Weichinger
  0 siblings, 1 reply; 16+ messages in thread
From: Alan McKinnon @ 2010-11-16 21:24 UTC (permalink / raw
  To: gentoo-user

Apparently, though unproven, at 23:12 on Tuesday 16 November 2010, Mick did 
opine thusly:

> > Don't think of --exclude as being a file path match, think of it as more
> > a regex (usually just a literal one). It specifies a pattern that if
> > found if the full pathname, results in the file not being synced.
> >
> > 
> >
> > The string "./System Volume Information" of course appears nowhere in the
> > rsync output list of files.
> >
> > 
> >
> > I think you want:
> > 
> >
> > --exclude "/System Volume Information"
> >
> > 
> >
> > Then rsync will not attempt to open the directory at all
> 
> Excellent, it worked!  :-)


Glad to hear it.

I could help because part of my job is running a rather big public ftp mirror 
that management graciously pay for. And I went down this rsync road a long 
time ago myself.

You have no idea how many brain cells died in agony to figure out this 
specific piece of rsync behaviour :-)


-- 
alan dot mckinnon at gmail dot com



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 21:24             ` Alan McKinnon
@ 2010-11-16 22:26               ` Stefan G. Weichinger
  2010-11-16 23:01                 ` Mick
  0 siblings, 1 reply; 16+ messages in thread
From: Stefan G. Weichinger @ 2010-11-16 22:26 UTC (permalink / raw
  To: gentoo-user

Am 2010-11-16 22:24, schrieb Alan McKinnon:
> Apparently, though unproven, at 23:12 on Tuesday 16 November 2010, Mick did 
> opine thusly:
>> Excellent, it worked!  :-)
> 
> 
> Glad to hear it.
> 
> I could help because part of my job is running a rather big public ftp mirror 
> that management graciously pay for. And I went down this rsync road a long 
> time ago myself.
> 
> You have no idea how many brain cells died in agony to figure out this 
> specific piece of rsync behaviour :-)

;-)

I would like to know if my suggestion also works ;-)

Yeah, include/exclude-patterns are rather hard to figure out sometimes
... nearly like regexes -> write once, read never ....

S



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 22:26               ` Stefan G. Weichinger
@ 2010-11-16 23:01                 ` Mick
  2010-11-25 21:51                   ` Renat Golubchyk
  0 siblings, 1 reply; 16+ messages in thread
From: Mick @ 2010-11-16 23:01 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: Text/Plain, Size: 1202 bytes --]

On Tuesday 16 November 2010 22:26:28 Stefan G. Weichinger wrote:
> Am 2010-11-16 22:24, schrieb Alan McKinnon:
> > Apparently, though unproven, at 23:12 on Tuesday 16 November 2010, Mick
> > did
> > 
> > opine thusly:
> >> Excellent, it worked!  :-)
> > 
> > Glad to hear it.
> > 
> > I could help because part of my job is running a rather big public ftp
> > mirror that management graciously pay for. And I went down this rsync
> > road a long time ago myself.
> > 
> > You have no idea how many brain cells died in agony to figure out this
> > specific piece of rsync behaviour :-)
> 
> ;-)
> 
> I would like to know if my suggestion also works ;-)
> 
> Yeah, include/exclude-patterns are rather hard to figure out sometimes
> ... nearly like regexes -> write once, read never ....

Ha, ha!  True!

Stefan, I tried escaping the spaces (even tried \\ double and \\\ triple 
escapes in case it makes a difference because of using ssh) but still did not 
work.  In my head I couldn't see how the full path would not work, but the 
relative path would, but I tried it out all the same.

I still don't understand why Alan's recommendation works   ;-)
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-16 23:01                 ` Mick
@ 2010-11-25 21:51                   ` Renat Golubchyk
  2010-11-26  8:02                     ` Helmut Jarausch
  2010-11-26 18:28                     ` Mick
  0 siblings, 2 replies; 16+ messages in thread
From: Renat Golubchyk @ 2010-11-25 21:51 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 3107 bytes --]

On Tue, 16 Nov 2010 23:01:51 +0000 Mick <michaelkintzios@gmail.com>
wrote:
> On Tuesday 16 November 2010 22:26:28 Stefan G. Weichinger wrote:
> > Am 2010-11-16 22:24, schrieb Alan McKinnon:
> > > Apparently, though unproven, at 23:12 on Tuesday 16 November
> > > 2010, Mick did
> > > 
> > > opine thusly:
> > >> Excellent, it worked!  :-)
> > > 
> > > Glad to hear it.
> > > 
> > > I could help because part of my job is running a rather big
> > > public ftp mirror that management graciously pay for. And I went
> > > down this rsync road a long time ago myself.
> > > 
> > > You have no idea how many brain cells died in agony to figure out
> > > this specific piece of rsync behaviour :-)
> > 
> > ;-)
> > 
> > I would like to know if my suggestion also works ;-)
> > 
> > Yeah, include/exclude-patterns are rather hard to figure out
> > sometimes ... nearly like regexes -> write once, read never ....
> 
> Ha, ha!  True!
> 
> Stefan, I tried escaping the spaces (even tried \\ double and \\\
> triple escapes in case it makes a difference because of using ssh)
> but still did not work.  In my head I couldn't see how the full path
> would not work, but the relative path would, but I tried it out all
> the same.
> 
> I still don't understand why Alan's recommendation works   ;-)

I'm probably late with my reply, but I'll post it so it will be in the
archives for future reference.

The man page is actually pretty clear on this issue. Quote:

  if the pattern starts with a / then it is anchored to a particular
  spot in the hierarchy of files, otherwise it  is  matched  against the
  end of the pathname.  This is similar to a leading ^ in regular
  expressions.  Thus "/foo" would match a name of "foo" at either the
  "root of the transfer" (for a global rule) or in the merge-file’s
  directory (for a per-directory  rule).   An  unqualified  "foo" would
  match  a  name  of "foo" anywhere in the tree because the algorithm is
  applied recursively from the top down; it behaves as if each path
  component gets a turn at being the end of the filename.  Even the
  unanchored "sub/foo" would match  at  any  point  in  the hierarchy
  where  a "foo" was found within a directory named "sub".

"Root of the transfer" is the directory you want to sync. Thus, if you
run e.g. "rsync /var/log/ /mnt/backups/ --exclude=/portage/" then root
of the transfer is /var/log, and therefore the directory
/var/log/portage will be excluded. If on the other hand you write
--exclude=portage/ then a directory named portage anywhere in the tree
under /var/log will be excluded. Without the trailing slash, i.e. just
--exclude=portage any file (regular file, directory, link, whatever)
named portage anywhere in the tree gets excluded. And finally
--exclude=/portage would exclude a file only at the top of the tree that
is going to be synchronsed.

Hope it helps.

Cheers,
Renat


-- 
Probleme kann man niemals mit derselben Denkweise loesen,
durch die sie entstanden sind.
                                              (Einstein)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-25 21:51                   ` Renat Golubchyk
@ 2010-11-26  8:02                     ` Helmut Jarausch
  2010-11-28 13:35                       ` Renat Golubchyk
  2010-11-26 18:28                     ` Mick
  1 sibling, 1 reply; 16+ messages in thread
From: Helmut Jarausch @ 2010-11-26  8:02 UTC (permalink / raw
  To: gentoo-user

On 11/25/10 22:51:36, Renat Golubchyk wrote:
> On Tue, 16 Nov 2010 23:01:51 +0000 Mick <michaelkintzios@gmail.com>
> wrote:
> > On Tuesday 16 November 2010 22:26:28 Stefan G. Weichinger wrote:
> > > Am 2010-11-16 22:24, schrieb Alan McKinnon:
> > > > Apparently, though unproven, at 23:12 on Tuesday 16 November
> > > > 2010, Mick did
> > > > 
> > > > opine thusly:
> > > >> Excellent, it worked!  :-)
> > > > 
> > > > Glad to hear it.
> > > > 
> > > > I could help because part of my job is running a rather big
> > > > public ftp mirror that management graciously pay for. And I 
> went
> > > > down this rsync road a long time ago myself.
> > > > 
> > > > You have no idea how many brain cells died in agony to figure
> out
> > > > this specific piece of rsync behaviour :-)
> > > 
> > > ;-)
> > > 
> > > I would like to know if my suggestion also works ;-)
> > > 
> > > Yeah, include/exclude-patterns are rather hard to figure out
> > > sometimes ... nearly like regexes -> write once, read never ....
> > 
> > Ha, ha!  True!
> > 
> > Stefan, I tried escaping the spaces (even tried \\ double and \\\
> > triple escapes in case it makes a difference because of using ssh)
> > but still did not work.  In my head I couldn't see how the full 
> path
> > would not work, but the relative path would, but I tried it out all
> > the same.
> > 
> > I still don't understand why Alan's recommendation works   ;-)
> 
> I'm probably late with my reply, but I'll post it so it will be in 
> the
> archives for future reference.
> 
> The man page is actually pretty clear on this issue. Quote:
> 
>   if the pattern starts with a / then it is anchored to a particular
>   spot in the hierarchy of files, otherwise it  is  matched  against
> the
>   end of the pathname.  This is similar to a leading ^ in regular
>   expressions.  Thus "/foo" would match a name of "foo" at either the
>   "root of the transfer" (for a global rule) or in the merge-file’s
>   directory (for a per-directory  rule).   An  unqualified  "foo"
> would
>   match  a  name  of "foo" anywhere in the tree because the algorithm
> is
>   applied recursively from the top down; it behaves as if each path
>   component gets a turn at being the end of the filename.  Even the
>   unanchored "sub/foo" would match  at  any  point  in  the hierarchy
>   where  a "foo" was found within a directory named "sub".
> 
> "Root of the transfer" is the directory you want to sync. Thus, if 
> you
> run e.g. "rsync /var/log/ /mnt/backups/ --exclude=/portage/" then 
> root
> of the transfer is /var/log, and therefore the directory
> /var/log/portage will be excluded. If on the other hand you write
> --exclude=portage/ then a directory named portage anywhere in the 
> tree
> under /var/log will be excluded. Without the trailing slash, i.e. 
> just
> --exclude=portage any file (regular file, directory, link, whatever)
> named portage anywhere in the tree gets excluded. And finally
> --exclude=/portage would exclude a file only at the top of the tree
> that
> is going to be synchronsed.
> 

Let me add some caveat which has trapped me recently.

I had (in your terms)
rsync /var/log/ /mnt/backups/ --delete --exclude=/portage/

and /var/log/portage was just a symlink to some other directory
while  /mnt/backups/portage was a real directory.
In that case rsync deletes /mnt/backups/portage !
It looks as if the "directory property" is check in the source tree
only. Quite an unpleasant surprise.

Helmut.




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-25 21:51                   ` Renat Golubchyk
  2010-11-26  8:02                     ` Helmut Jarausch
@ 2010-11-26 18:28                     ` Mick
  1 sibling, 0 replies; 16+ messages in thread
From: Mick @ 2010-11-26 18:28 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: Text/Plain, Size: 3218 bytes --]

On Thursday 25 November 2010 21:51:36 Renat Golubchyk wrote:
> On Tue, 16 Nov 2010 23:01:51 +0000 Mick <michaelkintzios@gmail.com>
> 
> wrote:
> > On Tuesday 16 November 2010 22:26:28 Stefan G. Weichinger wrote:
> > > Am 2010-11-16 22:24, schrieb Alan McKinnon:
> > > > Apparently, though unproven, at 23:12 on Tuesday 16 November
> > > > 2010, Mick did
> > > > 
> > > > opine thusly:
> > > >> Excellent, it worked!  :-)
> > > > 
> > > > Glad to hear it.
> > > > 
> > > > I could help because part of my job is running a rather big
> > > > public ftp mirror that management graciously pay for. And I went
> > > > down this rsync road a long time ago myself.
> > > > 
> > > > You have no idea how many brain cells died in agony to figure out
> > > > this specific piece of rsync behaviour :-)
> > > 
> > > ;-)
> > > 
> > > I would like to know if my suggestion also works ;-)
> > > 
> > > Yeah, include/exclude-patterns are rather hard to figure out
> > > sometimes ... nearly like regexes -> write once, read never ....
> > 
> > Ha, ha!  True!
> > 
> > Stefan, I tried escaping the spaces (even tried \\ double and \\\
> > triple escapes in case it makes a difference because of using ssh)
> > but still did not work.  In my head I couldn't see how the full path
> > would not work, but the relative path would, but I tried it out all
> > the same.
> > 
> > I still don't understand why Alan's recommendation works   ;-)
> 
> I'm probably late with my reply, but I'll post it so it will be in the
> archives for future reference.
> 
> The man page is actually pretty clear on this issue. Quote:
> 
>   if the pattern starts with a / then it is anchored to a particular
>   spot in the hierarchy of files, otherwise it  is  matched  against the
>   end of the pathname.  This is similar to a leading ^ in regular
>   expressions.  Thus "/foo" would match a name of "foo" at either the
>   "root of the transfer" (for a global rule) or in the merge-file’s
>   directory (for a per-directory  rule).   An  unqualified  "foo" would
>   match  a  name  of "foo" anywhere in the tree because the algorithm is
>   applied recursively from the top down; it behaves as if each path
>   component gets a turn at being the end of the filename.  Even the
>   unanchored "sub/foo" would match  at  any  point  in  the hierarchy
>   where  a "foo" was found within a directory named "sub".
> 
> "Root of the transfer" is the directory you want to sync. Thus, if you
> run e.g. "rsync /var/log/ /mnt/backups/ --exclude=/portage/" then root
> of the transfer is /var/log, and therefore the directory
> /var/log/portage will be excluded. If on the other hand you write
> --exclude=portage/ then a directory named portage anywhere in the tree
> under /var/log will be excluded. Without the trailing slash, i.e. just
> --exclude=portage any file (regular file, directory, link, whatever)
> named portage anywhere in the tree gets excluded. And finally
> --exclude=/portage would exclude a file only at the top of the tree that
> is going to be synchronsed.
> 
> Hope it helps.

Yes it does!  Thank you, it's clear to me now why it behaved so.

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [gentoo-user] How to exclude a directory from rsync
  2010-11-26  8:02                     ` Helmut Jarausch
@ 2010-11-28 13:35                       ` Renat Golubchyk
  0 siblings, 0 replies; 16+ messages in thread
From: Renat Golubchyk @ 2010-11-28 13:35 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 3295 bytes --]

On Fri, 26 Nov 2010 09:02:49 +0100 Helmut Jarausch
<jarausch@igpm.rwth-aachen.de> wrote:
> On 11/25/10 22:51:36, Renat Golubchyk wrote:
> > The man page is actually pretty clear on this issue. Quote:
> > 
> >   if the pattern starts with a / then it is anchored to a particular
> >   spot in the hierarchy of files, otherwise it  is  matched  against
> > the
> >   end of the pathname.  This is similar to a leading ^ in regular
> >   expressions.  Thus "/foo" would match a name of "foo" at either
> > the "root of the transfer" (for a global rule) or in the
> > merge-file’s directory (for a per-directory  rule).   An
> > unqualified  "foo" would
> >   match  a  name  of "foo" anywhere in the tree because the
> > algorithm is
> >   applied recursively from the top down; it behaves as if each path
> >   component gets a turn at being the end of the filename.  Even the
> >   unanchored "sub/foo" would match  at  any  point  in  the
> > hierarchy where  a "foo" was found within a directory named "sub".
> > 
> > "Root of the transfer" is the directory you want to sync. Thus, if 
> > you
> > run e.g. "rsync /var/log/ /mnt/backups/ --exclude=/portage/" then 
> > root
> > of the transfer is /var/log, and therefore the directory
> > /var/log/portage will be excluded. If on the other hand you write
> > --exclude=portage/ then a directory named portage anywhere in the 
> > tree
> > under /var/log will be excluded. Without the trailing slash, i.e. 
> > just
> > --exclude=portage any file (regular file, directory, link, whatever)
> > named portage anywhere in the tree gets excluded. And finally
> > --exclude=/portage would exclude a file only at the top of the tree
> > that
> > is going to be synchronsed.
> > 
> 
> Let me add some caveat which has trapped me recently.
> 
> I had (in your terms)
> rsync /var/log/ /mnt/backups/ --delete --exclude=/portage/
> 
> and /var/log/portage was just a symlink to some other directory
> while  /mnt/backups/portage was a real directory.
> In that case rsync deletes /mnt/backups/portage !
> It looks as if the "directory property" is check in the source tree
> only. Quite an unpleasant surprise.

The behavior is logical and consistent with the command you provided.
If you tell rsync to synchronise  two directories and delete everything
in the destination that is not in the source then rsync does just that.
And a symbolic link is not a directory. One might think that exclude
rule applies to both source and destination, but it doesn't. That's a
common pitfall. But the documentation is clear on that too. It says,

"This  option allows you to add rules to selectively exclude certain
files from the list of files to be transferred."

Since destination directory is not going to be transferred it is clear
that this option doesn't apply there.

In order to avoid the problem of files being accidentally deleted I
just never run rsync without running it with --dry-run first if I
specify a --delete option. Thus I have an opportunity to review the
file list before anything unpleasant is done.


Cheers,
Renat

-- 
Probleme kann man niemals mit derselben Denkweise loesen,
durch die sie entstanden sind.
                                              (Einstein)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2010-11-28 13:38 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-11-14 21:03 [gentoo-user] How to exclude a directory from rsync Mick
2010-11-14 22:47 ` Stefan G. Weichinger
2010-11-15 22:50   ` Mick
2010-11-16  9:00     ` Stefan G. Weichinger
2010-11-16 13:28       ` Mick
2010-11-16 14:15         ` Alan McKinnon
2010-11-16 21:12           ` Mick
2010-11-16 21:24             ` Alan McKinnon
2010-11-16 22:26               ` Stefan G. Weichinger
2010-11-16 23:01                 ` Mick
2010-11-25 21:51                   ` Renat Golubchyk
2010-11-26  8:02                     ` Helmut Jarausch
2010-11-28 13:35                       ` Renat Golubchyk
2010-11-26 18:28                     ` Mick
2010-11-16 14:36         ` Stefan G. Weichinger
2010-11-16 15:11         ` Robin Atwood

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox