[gentoo-user] OT: Copy directories in a special manner

[gentoo-user] OT: Copy directories in a special manner

[meino.cramer]

Hi,

 I want to do the following as fast as possible and with less
 system load as possible:

 There are two directories called 'source' and 'target'.

 'source' gets updated via 'svn up', then it gets compiled.
 Since there is no "make install" or similiar, installation
 is done via copying 'source' to 'target'. A symlink from
 /usr/local/bin/<name> to the compiled executabe in (now) 
 'target' completes the installation.

 BUT:
 'source' is a VERY big directory and copying it to 'target'
 after each svn up is a PAIN.

 Now I need a copy mechanism which does the following:
 * copy all files with newer date to 'target' -- this can be done via
   'cp -u'
 * copy all files which only exist in 'source' to 'target' --
   this can be done also with 'cp -u'
 * BUT: Delete all files from 'target' which do not longer exist in 
   'source'

 The last point gives me headaches. Scanning both directory after 
 'cp -u' has done its job may take as long as a "blind copy" from
 'source' to 'target' after 'target' was initially removed.

 Is there any "lean method" to do what is described above ?

 Thank you very much for any help in advance!
 Best regards,
 mcc

Re: [gentoo-user] OT: Copy directories in a special manner

[Paul Colquhoun]

On Sat, 5 Feb 2011 18:05:05 meino.cramer@gmx.de wrote:
> Hi,
> 
>  I want to do the following as fast as possible and with less
>  system load as possible:
> 
>  There are two directories called 'source' and 'target'.
> 
>  'source' gets updated via 'svn up', then it gets compiled.
>  Since there is no "make install" or similiar, installation
>  is done via copying 'source' to 'target'. A symlink from
>  /usr/local/bin/<name> to the compiled executabe in (now)
>  'target' completes the installation.
> 
>  BUT:
>  'source' is a VERY big directory and copying it to 'target'
>  after each svn up is a PAIN.
> 
>  Now I need a copy mechanism which does the following:
>  * copy all files with newer date to 'target' -- this can be done via
>    'cp -u'
>  * copy all files which only exist in 'source' to 'target' --
>    this can be done also with 'cp -u'
>  * BUT: Delete all files from 'target' which do not longer exist in
>    'source'
> 
>  The last point gives me headaches. Scanning both directory after
>  'cp -u' has done its job may take as long as a "blind copy" from
>  'source' to 'target' after 'target' was initially removed.
> 
>  Is there any "lean method" to do what is described above ?
> 
>  Thank you very much for any help in advance!
>  Best regards,
>  mcc


The tool you are looking for is 'rsync'.


-- 
Reverend Paul Colquhoun, ULC.    http://andor.dropbear.id.au/~paulcol
 Before you criticize someone, you should walk a mile in their shoes.
Then, when you do, you'll be a mile away, and you'll have their shoes.

[gentoo-user] Re: OT: Copy directories in a special manner

[Nikos Chantziaras]

On 02/05/2011 09:05 AM, meino.cramer@gmx.de wrote:
>   There are two directories called 'source' and 'target'.
>
>   'source' gets updated via 'svn up', then it gets compiled.
>   Since there is no "make install" or similiar, installation
>   is done via copying 'source' to 'target'. A symlink from
>   /usr/local/bin/<name>  to the compiled executabe in (now)
>   'target' completes the installation.
>
>   BUT:
>   'source' is a VERY big directory and copying it to 'target'
>   after each svn up is a PAIN.

Of course it's big, since it has all SVN data in it too.  You will find 
that after you delete that data with something like:

   rm -rf $(find . -name "*.svn")

It will become much smaller.

Re: [gentoo-user] OT: Copy directories in a special manner

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

On Sat, 5 Feb 2011 08:05:05 +0100, meino.cramer@gmx.de wrote:

>  Now I need a copy mechanism which does the following:
>  * copy all files with newer date to 'target' -- this can be done via
>    'cp -u'
>  * copy all files which only exist in 'source' to 'target' --
>    this can be done also with 'cp -u'
>  * BUT: Delete all files from 'target' which do not longer exist in 
>    'source'

rsync -a --delete source/ dest/


-- 
Neil Bothwick

Top Oxymorons Number 36: Alone together

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Re: OT: Copy directories in a special manner

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

Am 05.02.2011 10:06, schrieb Nikos Chantziaras:
> On 02/05/2011 09:05 AM, meino.cramer@gmx.de wrote:
>>   There are two directories called 'source' and 'target'.
>>
>>   'source' gets updated via 'svn up', then it gets compiled.
>>   Since there is no "make install" or similiar, installation
>>   is done via copying 'source' to 'target'. A symlink from
>>   /usr/local/bin/<name>  to the compiled executabe in (now)
>>   'target' completes the installation.
>>
>>   BUT:
>>   'source' is a VERY big directory and copying it to 'target'
>>   after each svn up is a PAIN.
> 
> Of course it's big, since it has all SVN data in it too.  You will find
> that after you delete that data with something like:
> 
>   rm -rf $(find . -name "*.svn")
> 
> It will become much smaller.
> 
> 

Easier (but untested!):
rsync -a --delete --exclude .svn/ source/ target

Excludes every directory called .svn from the transfer. Make sure to
include the slash after .svn or otherwise it matches files called .svn

Also take note that rsync changes its behavior depending on trailing
slashes on the source directory.

rsync -a source/ target
copies all content from source to target. For example source/foo ends up
as target/foo.

rsync -a source target
copies all content from source to target/source. For example source/foo
ends up as target/source/foo

Hope this helps,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] OT: Copy directories in a special manner

[meino.cramer]

Paul Colquhoun <paulcol@andor.dropbear.id.au> [11-02-05 10:08]:
> On Sat, 5 Feb 2011 18:05:05 meino.cramer@gmx.de wrote:
> > Hi,
> > 
> >  I want to do the following as fast as possible and with less
> >  system load as possible:
> > 
> >  There are two directories called 'source' and 'target'.
> > 
> >  'source' gets updated via 'svn up', then it gets compiled.
> >  Since there is no "make install" or similiar, installation
> >  is done via copying 'source' to 'target'. A symlink from
> >  /usr/local/bin/<name> to the compiled executabe in (now)
> >  'target' completes the installation.
> > 
> >  BUT:
> >  'source' is a VERY big directory and copying it to 'target'
> >  after each svn up is a PAIN.
> > 
> >  Now I need a copy mechanism which does the following:
> >  * copy all files with newer date to 'target' -- this can be done via
> >    'cp -u'
> >  * copy all files which only exist in 'source' to 'target' --
> >    this can be done also with 'cp -u'
> >  * BUT: Delete all files from 'target' which do not longer exist in
> >    'source'
> > 
> >  The last point gives me headaches. Scanning both directory after
> >  'cp -u' has done its job may take as long as a "blind copy" from
> >  'source' to 'target' after 'target' was initially removed.
> > 
> >  Is there any "lean method" to do what is described above ?
> > 
> >  Thank you very much for any help in advance!
> >  Best regards,
> >  mcc
> 
> 
> The tool you are looking for is 'rsync'.
> 
> 
> -- 
> Reverend Paul Colquhoun, ULC.    http://andor.dropbear.id.au/~paulcol
>  Before you criticize someone, you should walk a mile in their shoes.
> Then, when you do, you'll be a mile away, and you'll have their shoes.
> 

Hi all,

thank you very much for the hints!
Great! Saves me a lot of time.

Have a nice weekend!
Best regards
mcc