[gentoo-user] Writing a bash script or thinking about it anyway.

[gentoo-user] Writing a bash script or thinking about it anyway.

[Dale]

Hi folks,

Me again.  I'm thinking about writing a bash script that backs up my 
/home directory.  I found a guide but before I read all that stuff and 
muddy up the waters, is this thing current and will it work fine with 
the bash Gentoo uses?  Links to a even better guide would be good too.  
The guide I found is here:

http://tldp.org/LDP/abs/html/

I'm going to try to do this myself but as most of you know, it takes me 
a bit to grasp things.  I may be back for advice on this as well.  Who 
knows, maybe one day I can be a dev.  LOL  Well, most likely not really 
but anyway.

Thanks.

Dale

:-)  :-)

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Francisco Ares]

I probably didn't  get the point, but what about a simple "tar"?

#! /bin/bash
tar -cjvpf /tmp/home.tbz /home
mv /tmp/home.tbz /some/where/else

unless you're thinking on incremental back up, and more sophisticated things.

Francisco

On Tue, Dec 22, 2009 at 1:22 AM, Dale <rdalek1967@gmail.com> wrote:
> Hi folks,
>
> Me again.  I'm thinking about writing a bash script that backs up my /home
> directory.  I found a guide but before I read all that stuff and muddy up
> the waters, is this thing current and will it work fine with the bash Gentoo
> uses?  Links to a even better guide would be good too.  The guide I found is
> here:
>
> http://tldp.org/LDP/abs/html/
>
> I'm going to try to do this myself but as most of you know, it takes me a
> bit to grasp things.  I may be back for advice on this as well.  Who knows,
> maybe one day I can be a dev.  LOL  Well, most likely not really but anyway.
>
> Thanks.
>
> Dale
>
> :-)  :-)
>
>



-- 
"If you have an apple and I have an apple and we exchange apples then
you and I will still each have one apple. But if you have an idea and
I have one idea and we exchange these ideas, then each of us will have
two ideas." - George Bernard Shaw

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Dale]

Francisco Ares wrote:
> I probably didn't  get the point, but what about a simple "tar"?
>
> #! /bin/bash
> tar -cjvpf /tmp/home.tbz /home
> mv /tmp/home.tbz /some/where/else
>
> unless you're thinking on incremental back up, and more sophisticated things.
>
> Francisco
>
>   
Well, I want to start off making a small script.  Maybe get a little 
more complicated later on.  I do want to do incremental backups, at 
least at first.  I may later on use tar and something to keep say two 
copies and then delete the older ones. 

Just trying to get my feet wet here.  Trying to be simple at first and 
go from there.  If I try to cram to much in my head at one time, I get 
brain lock.

Thanks.

Dale

:-)  :-)

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Francisco Ares]

On Tue, Dec 22, 2009 at 2:16 AM, Dale <rdalek1967@gmail.com> wrote:
> Francisco Ares wrote:
>>
>> I probably didn't  get the point, but what about a simple "tar"?
>>
>> #! /bin/bash
>> tar -cjvpf /tmp/home.tbz /home
>> mv /tmp/home.tbz /some/where/else
>>
>> unless you're thinking on incremental back up, and more sophisticated
>> things.
>>
>> Francisco
>>
>>
>
> Well, I want to start off making a small script.  Maybe get a little more
> complicated later on.  I do want to do incremental backups, at least at
> first.  I may later on use tar and something to keep say two copies and then
> delete the older ones.
> Just trying to get my feet wet here.  Trying to be simple at first and go
> from there.  If I try to cram to much in my head at one time, I get brain
> lock.
>
> Thanks.
>
> Dale
>
> :-)  :-)
>
>

Well, there are a couple of tools to make your script quite simple:

http://www.linuxhowtos.org/Tips%20and%20Tricks/unison.htm
http://www.linuxjournal.com/article/7712
http://www.novell.com/coolsolutions/trench/16061.html
http://webtools.live2support.com/linux/rsync.php
http://www.linuxquestions.org/questions/linux-general-1/sync-a-usb-flash-drive-with-hd-folders-possible-522875/
http://www.linux.com/news/enterprise/storage/8200-back-up-like-an-expert-with-rsync
http://www.unixtutorial.org/2008/09/how-to-synchronize-directories-with-rsync/

Hope this helps
Francisco

-- 
"If you have an apple and I have an apple and we exchange apples then
you and I will still each have one apple. But if you have an idea and
I have one idea and we exchange these ideas, then each of us will have
two ideas." - George Bernard Shaw

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Neil Walker]

Dale wrote:
> Me again.  I'm thinking about writing a bash script that backs up my
> /home directory.

I use a simple rsync cron job to backup entire servers every hour. Does
the job for me. ;)


Be lucky,

Neil
http://www.the-workathome.com

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Dale]

Neil Walker wrote:
> Dale wrote:
>   
>> Me again.  I'm thinking about writing a bash script that backs up my
>> /home directory.
>>     
>
> I use a simple rsync cron job to backup entire servers every hour. Does
> the job for me. ;)
>
>
> Be lucky,
>
> Neil
> http://www.the-workathome.com
>
>   

But I wouldn't learn how to write a script that way.  I got to start 
somewhere.  This is a good place.

Dale

:-)  :-) 

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Francisco Ares]

On Tue, Dec 22, 2009 at 3:09 AM, Dale <rdalek1967@gmail.com> wrote:
> Neil Walker wrote:
>>
>> Dale wrote:
>>
>>>
>>> Me again.  I'm thinking about writing a bash script that backs up my
>>> /home directory.
>>>
>>
>> I use a simple rsync cron job to backup entire servers every hour. Does
>> the job for me. ;)
>>
>>
>> Be lucky,
>>
>> Neil
>> http://www.the-workathome.com
>>
>>
>
> But I wouldn't learn how to write a script that way.  I got to start
> somewhere.  This is a good place.
>
> Dale
>
> :-)  :-)
>

Now I got your point.

I would think on something like this (untested):


The command line would be

BackupScriptName  FromDirectory  ToDirectory


#! /bin/bash
FROM=$1
TO=$2
cd $FROM
for i in * .??*
do
    if [[ -d "$i" ]]   # is it a directory?
    then
        # yes, it is a directory
        if [[ -e "$TO/$i" ]] && [[ -d "$TO/$i" ]]
        then

        # on the TO side, the name exists
        # and it is a directory
            cd "$i"

            # just to show something
            pwd

            # calls recursively this same script
            $0 "$FROM/$i" "$TO/$i"
            cd ..

        else
            # didn't existed yet on the TO side,
            # so copy everything
            cp -a $FROM/$i $TO/$i
        fi
    else
        # it is a file, not a directory
        if [[ -e "$TO/$i" ]]
        then
            # the file already exists

            # do something to compare the files, like:

            # gets size from "ls" result
            SIZE1=`ls -l "$i" | cut -d" " -f5`
            SIZE2=`ls -l "TO/$i" | cut -d" " -f5

            if (( $SIZE1!=$SIZE2 ))
            then

                # size is different, so copy the file
                cp -a "$FROM/$i" "$TO/$i"

            else
                # more tests for differences other
                # than size, like date/time
                # or even MD5SUM
            fi
        else

            # file doesn't exist at the "TO" side
            cp -a "$FROM/$i" "$TO/$i"

        fi

    fi

done


Hope this helps
Francisco
-- 
"If you have an apple and I have an apple and we exchange apples then
you and I will still each have one apple. But if you have an idea and
I have one idea and we exchange these ideas, then each of us will have
two ideas." - George Bernard Shaw

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Dale]

Francisco Ares wrote:
> On Tue, Dec 22, 2009 at 3:09 AM, Dale <rdalek1967@gmail.com> wrote:
>   
>> Neil Walker wrote:
>>     
>>> Dale wrote:
>>>
>>>       
>>>> Me again.  I'm thinking about writing a bash script that backs up my
>>>> /home directory.
>>>>
>>>>         
>>> I use a simple rsync cron job to backup entire servers every hour. Does
>>> the job for me. ;)
>>>
>>>
>>> Be lucky,
>>>
>>> Neil
>>> http://www.the-workathome.com
>>>
>>>
>>>       
>> But I wouldn't learn how to write a script that way.  I got to start
>> somewhere.  This is a good place.
>>
>> Dale
>>
>> :-)  :-)
>>
>>     
>
> Now I got your point.
>
> I would think on something like this (untested):
>
>
> The command line would be
>
> BackupScriptName  FromDirectory  ToDirectory
>
>
> #! /bin/bash
> FROM=$1
> TO=$2
> cd $FROM
> for i in * .??*
> do
>     if [[ -d "$i" ]]   # is it a directory?
>     then
>         # yes, it is a directory
>         if [[ -e "$TO/$i" ]] && [[ -d "$TO/$i" ]]
>         then
>
>         # on the TO side, the name exists
>         # and it is a directory
>             cd "$i"
>
>             # just to show something
>             pwd
>
>             # calls recursively this same script
>             $0 "$FROM/$i" "$TO/$i"
>             cd ..
>
>         else
>             # didn't existed yet on the TO side,
>             # so copy everything
>             cp -a $FROM/$i $TO/$i
>         fi
>     else
>         # it is a file, not a directory
>         if [[ -e "$TO/$i" ]]
>         then
>             # the file already exists
>
>             # do something to compare the files, like:
>
>             # gets size from "ls" result
>             SIZE1=`ls -l "$i" | cut -d" " -f5`
>             SIZE2=`ls -l "TO/$i" | cut -d" " -f5
>
>             if (( $SIZE1!=$SIZE2 ))
>             then
>
>                 # size is different, so copy the file
>                 cp -a "$FROM/$i" "$TO/$i"
>
>             else
>                 # more tests for differences other
>                 # than size, like date/time
>                 # or even MD5SUM
>             fi
>         else
>
>             # file doesn't exist at the "TO" side
>             cp -a "$FROM/$i" "$TO/$i"
>
>         fi
>
>     fi
>
> done
>
>
> Hope this helps
> Francisco
>   

After learning a little about scripting I'll give this a once over and 
see what I learn.  Right now, it's like. . . huh.  lol

Thanks.

Dale

:-)  :-) 

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Jesús Guerrero]

On Mon, 21 Dec 2009 21:22:05 -0600, Dale <rdalek1967@gmail.com> wrote:
> Hi folks,
> 
> Me again.  I'm thinking about writing a bash script that backs up my 
> /home directory.  I found a guide but before I read all that stuff and 
> muddy up the waters, is this thing current and will it work fine with 
> the bash Gentoo uses?  Links to a even better guide would be good too.  
> The guide I found is here:
> 
> http://tldp.org/LDP/abs/html/

The advanced bash scripting guide will be equally valid nowadays as it was
when it was first written. A few minor edges have changed in bash in the
latest times, but you are unlikely to get touched by these unless you are
using some rare feature. After all, bash is compatible with the original
bourne shell to a big extent, and that part of its behavior never changes.

If you truly want to learn bash, I say go for it and come back when/if you
have some problem.

-- 
Jesús Guerrero

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Christian Könitzer]

I agree with Jesús but recommend you to use rsync for backup purpose.
Simple google for rsync backup script.
And this link explains why:
http://www.sanitarium.net/golug/rsync_backups.html

There are a lot backup scripts using rsync out there and most of them 
are written in bash so it's anyway a good idea to learn a bit bash ;)


Am 22.12.2009 11:34, schrieb Jesús Guerrero:
> On Mon, 21 Dec 2009 21:22:05 -0600, Dale<rdalek1967@gmail.com>  wrote:
>    
>> Hi folks,
>>
>> Me again.  I'm thinking about writing a bash script that backs up my
>> /home directory.  I found a guide but before I read all that stuff and
>> muddy up the waters, is this thing current and will it work fine with
>> the bash Gentoo uses?  Links to a even better guide would be good too.
>> The guide I found is here:
>>
>> http://tldp.org/LDP/abs/html/
>>      
> The advanced bash scripting guide will be equally valid nowadays as it was
> when it was first written. A few minor edges have changed in bash in the
> latest times, but you are unlikely to get touched by these unless you are
> using some rare feature. After all, bash is compatible with the original
> bourne shell to a big extent, and that part of its behavior never changes.
>
> If you truly want to learn bash, I say go for it and come back when/if you
> have some problem.
>
>    

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Jesús Guerrero]

On Tue, 22 Dec 2009 12:20:46 +0100, Christian Könitzer <chkab@gmx.ch>
wrote:
> I agree with Jesús but recommend you to use rsync for backup purpose.
> Simple google for rsync backup script.
> And this link explains why:
> http://www.sanitarium.net/golug/rsync_backups.html
> 
> There are a lot backup scripts using rsync out there and most of them 
> are written in bash so it's anyway a good idea to learn a bit bash ;)

That's basically it. I also agree that rsync is nowadays the way to go for
general purpose backups, unless...

  a) the volume can't be mounted while the backup is running (not the
case)
  b) your fs supports snapshots (or you use LVM)
  c) you have an rcs based solution, like svn, git or whatever else

The snapshot option is the absolute safest because it ensures that all the
files will be consistent, and the tar based solution is probably the worst
of them unless you truly can guarantee that the files are not being written
concurrently while you do the backup (i.e. the fs is not being used at all
or it's mounted read only). A good practice in any case is to capture the
exist status of any given tool you are going to use to ensure that no error
happened. Discovering that a backup is incomplete or corrupt when you need
it is a bit unpleasant to say the least.

Bash can be used for that, just to put a simple example. As you say,
regardless of the solution of your choice to do the actual backup there's a
lot of room for improvement, automation, error loging, verification, etc.
using shell scripting.


-- 
Jesús Guerrero

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Christian Könitzer]

a question to b):
Can you tell me a fs that supports snappshots (I'm planing to set up a 
new server so you can choose a new fs... (now I am using reiserfs)) and 
maybe how to use it (link)? So if you say "or LSM" does this mean I can 
achieve this also woth LVM? How?
thx...

If you have some more tips, my current server runs on a single HDD and a 
daily rsync to another computer makes my backup. In my new server I'd 
like to improve as much as possible for as less money and energy as 
possible (since it's only for me...) so I thought about RAID as well. 
Today I'm using very old standard desktop hardware and for the new one I 
like to use also standard hardware so I don't know if I can afford a 
RAID-5 card or so... and I never made software RAID... do you have some 
recommendations (links?)? thx

cheers
Christian

Am 22.12.2009 13:11, schrieb Jesús Guerrero:
> On Tue, 22 Dec 2009 12:20:46 +0100, Christian Könitzer<chkab@gmx.ch>
> wrote:
>    
>> I agree with Jesús but recommend you to use rsync for backup purpose.
>> Simple google for rsync backup script.
>> And this link explains why:
>> http://www.sanitarium.net/golug/rsync_backups.html
>>
>> There are a lot backup scripts using rsync out there and most of them
>> are written in bash so it's anyway a good idea to learn a bit bash ;)
>>      
> That's basically it. I also agree that rsync is nowadays the way to go for
> general purpose backups, unless...
>
>    a) the volume can't be mounted while the backup is running (not the
> case)
>    b) your fs supports snapshots (or you use LVM)
>    c) you have an rcs based solution, like svn, git or whatever else
>
> The snapshot option is the absolute safest because it ensures that all the
> files will be consistent, and the tar based solution is probably the worst
> of them unless you truly can guarantee that the files are not being written
> concurrently while you do the backup (i.e. the fs is not being used at all
> or it's mounted read only). A good practice in any case is to capture the
> exist status of any given tool you are going to use to ensure that no error
> happened. Discovering that a backup is incomplete or corrupt when you need
> it is a bit unpleasant to say the least.
>
> Bash can be used for that, just to put a simple example. As you say,
> regardless of the solution of your choice to do the actual backup there's a
> lot of room for improvement, automation, error loging, verification, etc.
> using shell scripting.
>
>
>    

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Alan McKinnon]

On Tuesday 22 December 2009 16:21:08 Christian Könitzer wrote:
> a question to b):
> Can you tell me a fs that supports snappshots (I'm planing to set up a
> new server so you can choose a new fs... (now I am using reiserfs)) and
> maybe how to use it (link)? So if you say "or LSM" does this mean I can
> achieve this also woth LVM? How?
> thx...

None of the traditional filesystems (ext2|3, reiser) support snapshots. ZFS< 
Btrfs do, possibly ext4 also (the last is a hunch only).

LVM snapshots a volume, not the filesystem on it. So it tracks extents that 
have changed, not individual files. For backup purposes though, volume and fs 
snapshots are equivalent.

Snapshots with LVM are easy as pie:

- create a new volume which is a snapshot of an existing one
- mount the snapshot somewhere
- copy,backup,etc as you like. The volume is read-only so you can't break it
- umount snapshot
- destroy snapshot

The LVM man pages contain a wealth of data, as does Google and the LVM 
documentation at redhat.com

> 
> If you have some more tips, my current server runs on a single HDD and a
> daily rsync to another computer makes my backup. In my new server I'd
> like to improve as much as possible for as less money and energy as
> possible (since it's only for me...) so I thought about RAID as well.
> Today I'm using very old standard desktop hardware and for the new one I
> like to use also standard hardware so I don't know if I can afford a
> RAID-5 card or so... and I never made software RAID... do you have some
> recommendations (links?)? thx

Cheap dumbass hardware raid is not worth the money. This includes every single 
on-board raid controlled it has ever been my misfortune to behold. Some of 
them are even called RAID but aren't - they're just a fancy multipath thingy 
and might let you only choose RAID 1 or 5, or even only RAID 1 in some cases

Software raid built into the linux kernel is a far better solution. Again, 
Google will return enough hits that it will take you a year to read all the 
good docs....

Proper hardware add-on cards *are* worth the money though and deliver great 
results. Your post indicates that you won't want to spend what they cost 
though

-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Jesús Guerrero]

On Tue, 22 Dec 2009 16:51:08 +0200, Alan McKinnon
<alan.mckinnon@gmail.com>
wrote:
> On Tuesday 22 December 2009 16:21:08 Christian Könitzer wrote:
>> a question to b):
>> Can you tell me a fs that supports snappshots (I'm planing to set up a
>> new server so you can choose a new fs... (now I am using reiserfs)) and
>> maybe how to use it (link)? So if you say "or LSM" does this mean I can
>> achieve this also woth LVM? How?
>> thx...
> 
> None of the traditional filesystems (ext2|3, reiser) support snapshots.
> ZFS< 
> Btrfs do, possibly ext4 also (the last is a hunch only).

That's basically true. However btrfs is quite experimental still, and I
have no serious experience with ZFS, it kind of turns me back the fact that
it's a FUSE based fs, though it's certainly possible to use it even for a
root system provided that your kernel can load the module at bootup
(initrd), I have no idea if there's any downside. I don't have either any
notice about snapshotting in ext4 (I remember the plan being discussed but
I don't think it has been finally implemented, I'd like to be wrong on this
one though).

> LVM snapshots a volume, not the filesystem on it. So it tracks extents
> that 
> have changed, not individual files. For backup purposes though, volume
and
> fs 
> snapshots are equivalent.
> 
> Snapshots with LVM are easy as pie:
> 
> - create a new volume which is a snapshot of an existing one
> - mount the snapshot somewhere
> - copy,backup,etc as you like. The volume is read-only so you can't
break
> it
> - umount snapshot
> - destroy snapshot
> 
> The LVM man pages contain a wealth of data, as does Google and the LVM 
> documentation at redhat.com

Yep, just googling for something along the lines of "lvm snapshot backup"
should give you enough info to start researching. However, for this to be a
possibility you first need to convert your system to use lvm.

-- 
Jesús Guerrero

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Stroller]

On 22 Dec 2009, at 15:36, Jesús Guerrero wrote:
> ... I have no serious experience with ZFS, it kind of turns me back  
> the fact that
> it's a FUSE based fs, though it's certainly possible to use it even  
> for a
> root system provided that your kernel can load the module at  
> bootup ...

ZFS shouldn't be considered a FUSE-based filesystem. It should be  
considered a Solaris filesystem, which is yet to be (??) adequately  
ported to any other platforms.

ZFS on Linux would be lovely, but Btrfs will do just fine when it's  
done. IIRC Btrfs boasts some features that ZFS does not, and lacks  
some others.

Stroller.

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Dirk Heinrichs]

Am Dienstag 22 Dezember 2009 20:01:16 schrieb Stroller:
> ZFS shouldn't be considered a FUSE-based filesystem. It should be  
> considered a Solaris filesystem, which is yet to be (??) adequately  
> ported to any other platforms.

It just left experimental state in FreeBSD 8.0.

Bye...

	Dirk

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Stroller]

On 22 Dec 2009, at 10:34, Jesús Guerrero wrote:
>> ...
>> http://tldp.org/LDP/abs/html/
>
> The advanced bash scripting guide will be equally valid nowadays as  
> it was
> when it was first written.

It seems well-maintained from the revision history at the top of the  
linked page.

Stroller.

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Willie Wong]

On Tue, Dec 22, 2009 at 11:34:34AM +0100, Penguin Lover Jes??s Guerrero squawked:
> The advanced bash scripting guide will be equally valid nowadays as it was
> when it was first written. A few minor edges have changed in bash in the
> latest times, but you are unlikely to get touched by these unless you are
> using some rare feature. After all, bash is compatible with the original
> bourne shell to a big extent, and that part of its behavior never changes.

Really? I had quite a few scripts die on me in the Bash 3.1-3.2
change. One of the problems is complained about enough that they now
introduced a compatibility level (item l on the ChangeLog
  http://tiswww.case.edu/php/chet/bash/NEWS )

But given that the ABS was last updated this year, I should say
anything in it is up-to-date enough. 

W
-- 
You're not paranoid.
The world _IS_ fucked.
Sortir en Pantoufles: up 1110 days, 22:12

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Stroller]

On 22 Dec 2009, at 03:22, Dale wrote:
> ... I found a guide but before I read all that stuff and muddy up  
> the waters, is this thing current and will it work fine with the  
> bash Gentoo uses?  Links to a even better guide would be good too.   
> The guide I found is here:
>
> http://tldp.org/LDP/abs/html/

The Advanced Scripting Guide is brilliant. It's my gospel when Bash  
scripting, although I'll admit I haven't looked harder for anything  
else. If there is any better guide out there, I would love to hear  
about it.

The examples given are just that - examples. I'm sure they're intended  
for you to make changes in order to suit yourself, although you may  
well find that some of them suit you just fine as is.

Stroller.

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Willie Wong]

On Tue, Dec 22, 2009 at 11:16:36AM +0000, Penguin Lover Stroller squawked:
> On 22 Dec 2009, at 03:22, Dale wrote:
>> ... I found a guide but before I read all that stuff and muddy up the 
>> waters, is this thing current and will it work fine with the bash Gentoo 
>> uses?  Links to a even better guide would be good too.  The guide I found 
>> is here:
>>
>> http://tldp.org/LDP/abs/html/
>
> The Advanced Scripting Guide is brilliant. It's my gospel when Bash 
> scripting, although I'll admit I haven't looked harder for anything else. 
> If there is any better guide out there, I would love to hear about it.

I agree. For learning the ropes there's the ABS, for everything else
there is the man/info pages. (Learning Bash was the reason that I
learned gnu info. That one man page is just way too long.)

W
-- 
Ever stop to think, and forget to start again?
Sortir en Pantoufles: up 1110 days, 22:19

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Ward Poelmans]

On Tue, Dec 22, 2009 at 04:22, Dale <rdalek1967@gmail.com> wrote:
> Hi folks,
>
> Me again.  I'm thinking about writing a bash script that backs up my /home
> directory.  I found a guide but before I read all that stuff and muddy up
> the waters, is this thing current and will it work fine with the bash Gentoo
> uses?

You should take a look at rsnapshot ( http://rsnapshot.org/ ). I use
it and it works perfectly. But it's written in perl and not bash, i'm
afraid.

Ward

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1154 bytes --]

On Tue, 22 Dec 2009 16:08:18 +0100, Ward Poelmans wrote:

> You should take a look at rsnapshot ( http://rsnapshot.org/ ). I use
> it and it works perfectly. But it's written in perl and not bash, i'm
> afraid.

That misses the point. This isn't really about making backups, it's about
learning to write shell scripts. Backing up is just the project that Dale
has chosen for this first effort, at least that's how I read it.

Having said that, you need an underlying program to perform the backups,
and the main choices are tar and rsync. Rsync has all the advantages
already mentioned, but it does need a lot of space and needs the
filesystem to be used for backups to support all the metadata of the
source. Tar handles both of these issues, but incremental backups are
more complicated to make and restore. On the other hand, you do get
history, while rsync only has the most recent copy of the file. If you
found the file was corrupt just after an rsync backup, touch!

I really like BackupPC, but that's totally irrelevant to the original
question.


-- 
Neil Bothwick

We all know what comes after 'X', said Tom, wisely.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Ward Poelmans]

On Tue, Dec 22, 2009 at 16:39, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Tue, 22 Dec 2009 16:08:18 +0100, Ward Poelmans wrote:
>
>> You should take a look at rsnapshot ( http://rsnapshot.org/ ). I use
>> it and it works perfectly. But it's written in perl and not bash, i'm
>> afraid.
>
> That misses the point. This isn't really about making backups, it's about
> learning to write shell scripts. Backing up is just the project that Dale
> has chosen for this first effort, at least that's how I read it.

Sorry, i've read the original message a bit too fast. But it's still
worth to take a look at how rsnapshot works if you want to write
anything yourself. I think that the combination of rsync and hardlinks
is quite effective and fast.

Ward

[gentoo-user] Re: Writing a bash script or thinking about it anyway.

[»Q«]

On Mon, 21 Dec 2009 21:22:05 -0600
Dale <rdalek1967@gmail.com> wrote:

> I found a guide but before I read all that stuff and muddy up the
> waters, is this thing current and will it work fine with the bash
> Gentoo uses?  Links to a even better guide would be good too. The
> guide I found is here:
> 
> http://tldp.org/LDP/abs/html/

It's current and it's fine for Gentoo.  It's in the portage tree as
app-doc/abs-guide, in case you want a local copy.

-- 
»Q«
     Kleeneness is next to Gödelness.

Re: [gentoo-user] Re: Writing a bash script or thinking about it anyway.

[Dale]

»Q« wrote:
> On Mon, 21 Dec 2009 21:22:05 -0600
> Dale <rdalek1967@gmail.com> wrote:
>
>   
>> I found a guide but before I read all that stuff and muddy up the
>> waters, is this thing current and will it work fine with the bash
>> Gentoo uses?  Links to a even better guide would be good too. The
>> guide I found is here:
>>
>> http://tldp.org/LDP/abs/html/
>>     
>
> It's current and it's fine for Gentoo.  It's in the portage tree as
> app-doc/abs-guide, in case you want a local copy.
>
>   

I'm installing this and will give it a read when I get some free time.  
I hope this will not be to long. 

Thanks.

Dale

:-)  :-)

[gentoo-user] Re: Writing a bash script or thinking about it anyway.

[»Q«]

On Fri, 25 Dec 2009 17:19:37 -0600
Dale <rdalek1967@gmail.com> wrote:

> »Q« wrote:

> > It's current and it's fine for Gentoo.  It's in the portage tree as
> > app-doc/abs-guide, in case you want a local copy.
> 
> I'm installing this and will give it a read when I get some free
> time. I hope this will not be to long.

I'd forgotten about this, but drobbins wrote a series of three "Bash by
Example" articles almost ten years ago, and the first two cover a lot
of the fundamental stuff.  I found them useful when I was starting
out.  (Heh, I'm not sure I'm done starting out.)

<http://www.ibm.com/developerworks/library/l-bash.html>

Some of the Gentoo-specific things in the third article aren't true
any more, for example, "Bash is an essential component of the Gentoo
Linux ebuild system."

-- 
»Q«
     Kleeneness is next to Gödelness.

Re: [gentoo-user] Writing a bash script or thinking about it anyway.

[Alex Schuster]

Dale writes:

> Me again.  I'm thinking about writing a bash script that backs up my
> /home directory.  I found a guide but before I read all that stuff and
> muddy up the waters, is this thing current and will it work fine with
> the bash Gentoo uses?  Links to a even better guide would be good too.
> The guide I found is here:
> 
> http://tldp.org/LDP/abs/html/

I learnt bash that way. And by reading the man page over and over again. 
And again. Now I have scripts > 100k in size, I had not expected that bash 
was so powerful.

http://www.shelldorado.com/shelltips/ has some nice tips
http://sayle.net/book/basics.htm#what_is_a_shell explains what a shell is, 
maybe that's something to browse first.

But for the backup: I recommend rdiff-backup and rsnapshot, I use the 
former. Basically, it rsyncs a directory, but allows incremental backups, 
too. After the first backup, the backup directory looks exactly like the 
original directory, except for a 'rdiff-backup-data' folder containing 
additional information. When A drive went bad, I just renamed my 1st and 
2nd drives LVM volume groups and rebooted, now my system ran on the backup 
drive.
The next backups only back up files that have been modified, and save the 
compressed differences into the rsync-backup-data folder.

rsnapshot works similar (but I did not use it yet). The main difference is 
that rsnapshot creates a whole directory for every backup, containing 
exactly the files as they were present at the time of backup. Files that 
stayed the same are not copied, but hard-linked so they take up no extra 
space.

So, I would say, rsnapshot is more practical, at the expense of storage 
space. If you need a file from an older backup, juts look into the 
corresponding directory, it is there, just copy it over, no need to use a 
backup tool to restore stuff. If you want to save space, try rdiff-backup. 
But if you need a file not from the last backup, but from an earlier one, 
you need to extract it first with rsnapshot (No big deal, still).

My backup script was written in bash of course. It has some targets like 
'home', 'usr', 'opt' (backup this partition) and extras like 'src' (create 
.tbz2 files of /usr/src/linux-*), 'etc' (tar /etc) or 'kde' (tar ~/.kde*, 
I regularly do this before I save the session, because this often does not 
work). All my data is on LVM volumes, so when a partition is to be backed 
up, a LVM snapshot is created and used for the backup. So I can continue 
working in my home directory and change data during the backup.
The script is here: http://wonkology.org/~wonko/tmp/backup

	Wonko