[gentoo-user] new machine: incremental back-ups

[gentoo-user] new machine: incremental back-ups

[Philip Webb]

My new machine is working very well -- thanks again for the advice -- ,
tho' the sighing noise from the CPU fan is a bit trying (smile),
& I'm considering setting up an incremental back-up system
so that if the SSD collapses,
I can restore everything from files stored on the HDD without re-installing.

I already have an adequate set-up of back-ups, incl off-site copies,
which ensures my personal files + configs cb restored,
but it involves some human intervention from me at various times
& I would have to re-install all the system + Portage stuff.

What do people who do incremental back-ups use ?

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] new machine: incremental back-ups

[Michael Hampicke]

Am 04.10.2012 08:38, schrieb Philip Webb:
> My new machine is working very well -- thanks again for the advice -- ,
> tho' the sighing noise from the CPU fan is a bit trying (smile),
> & I'm considering setting up an incremental back-up system
> so that if the SSD collapses,
> I can restore everything from files stored on the HDD without re-installing.
> 
> I already have an adequate set-up of back-ups, incl off-site copies,
> which ensures my personal files + configs cb restored,
> but it involves some human intervention from me at various times
> & I would have to re-install all the system + Portage stuff.
> 
> What do people who do incremental back-ups use ?
> 

I am using http://backuppc.sourceforge.net/ (in portage) which brings
some nice features like compression, de-duplication and a web interface.
Once configured it runs automatically.

Just as of today I started some tests with storeBackup (which is not in
portage: http://www.nongnu.org/storebackup/). It has a nice feature
list: compression, de-dup (even on block devices), backup replication,
checksums and the author claims, you can restore backups even without
having storeBackup at hand.

What do you use for cloud/offsite backups? I am still searching for the
perfect solution. Requirements: backups must be encrypted, delta-sync,
and being able to resume interrupted transfers to the offsite location.

At the moment I am using duply, which is a frontend for duplictiy. But
with duplicity, a flipped bit in the backup chain can render all
following backups useless.
So now I am thinking about encfs. Just encrypt the backup (or use encfs
--reverse) and rsync to offsite location.

Re: [gentoo-user] new machine: incremental back-ups

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 836 bytes --]

On Thu, 04 Oct 2012 10:25:53 +0200, Michael Hampicke wrote:

> I am using http://backuppc.sourceforge.net/ (in portage) which brings
> some nice features like compression, de-duplication and a web interface.
> Once configured it runs automatically.

+1 for BackupPC

> What do you use for cloud/offsite backups? I am still searching for the
> perfect solution. Requirements: backups must be encrypted, delta-sync,
> and being able to resume interrupted transfers to the offsite location.

I have a Python script that uses a combination of dar,
to create encrypted backups locally, and boto to upload them to S3. I used
duplicity some years ago and found it consumed enormous amounts of
bandwidth, more than my ISP provided at the time.


-- 
Neil Bothwick

If a parsley farmer is sued, can they garnish his wages?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] new machine: incremental back-ups

[Michael Hampicke]

Am 04.10.2012 12:37, schrieb Neil Bothwick:
> On Thu, 04 Oct 2012 10:25:53 +0200, Michael Hampicke wrote:
> 
>> I am using http://backuppc.sourceforge.net/ (in portage) which brings
>> some nice features like compression, de-duplication and a web interface.
>> Once configured it runs automatically.
> 
> +1 for BackupPC
> 
>> What do you use for cloud/offsite backups? I am still searching for the
>> perfect solution. Requirements: backups must be encrypted, delta-sync,
>> and being able to resume interrupted transfers to the offsite location.
> 
> I have a Python script that uses a combination of dar,
> to create encrypted backups locally, and boto to upload them to S3. I used
> duplicity some years ago and found it consumed enormous amounts of
> bandwidth, more than my ISP provided at the time.

Hm, dar looks interesting. I'll have a look at it. The man page states
that it is possible to restore individual files from an dar archive
without reading the complete file (in contrast to tar). Is this also
true when using compression and/or encryption? Would be a great feature
for fast single file restores (Just mount the offsite location with
sshfs or similar and tell dar to restore file25 from my 500GB backup
without having to transfer the whole damn thing :) )

Re: [gentoo-user] new machine: incremental back-ups

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 873 bytes --]

On Thu, 04 Oct 2012 13:43:13 +0200, Michael Hampicke wrote:

> Hm, dar looks interesting. I'll have a look at it. The man page states
> that it is possible to restore individual files from an dar archive
> without reading the complete file (in contrast to tar). Is this also
> true when using compression and/or encryption? Would be a great feature
> for fast single file restores (Just mount the offsite location with
> sshfs or similar and tell dar to restore file25 from my 500GB backup
> without having to transfer the whole damn thing :) )

The way I do it is to use the split feature to split the archive into
manageable chunks. It helps when an upload fails and when restoring you
just grab the catalog, try to restore from that and it tells you which
file(s) it needs. 


-- 
Neil Bothwick

People who eat natural foods die from natural causes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] new machine: incremental back-ups

[William Kenworthy]

On Thu, 2012-10-04 at 02:38 -0400, Philip Webb wrote:
> My new machine is working very well -- thanks again for the advice -- ,
> tho' the sighing noise from the CPU fan is a bit trying (smile),
> & I'm considering setting up an incremental back-up system
> so that if the SSD collapses,
> I can restore everything from files stored on the HDD without re-installing.
> 
> I already have an adequate set-up of back-ups, incl off-site copies,
> which ensures my personal files + configs cb restored,
> but it involves some human intervention from me at various times
> & I would have to re-install all the system + Portage stuff.
> 
> What do people who do incremental back-ups use ?
> 

dirvish - its in portage

"In other news, I've learned from the director of the Oregon State
University Open Source Lab that they will be backing up their servers
with dirvish. These servers are the primary mirror sites for Mozilla,
Kernel.org, Gentoo, Drupal, and other major open source projects."

This is from the dirvish website in 2005 when I started using it - if
its good enough for those guys ...

I am still using it on multiple systems without problems.  Space
efficient versioned backups.  The website is a bit out of date, but
there is still dev going on behind the scenes as shown by googling -
solid and reliable so doesnt need to change much.

Recommended.

BillK

Re: [gentoo-user] new machine: incremental back-ups

[Paul Hartman]

On Thu, Oct 4, 2012 at 1:38 AM, Philip Webb <purslow@ca.inter.net> wrote:
> My new machine is working very well -- thanks again for the advice -- ,
> tho' the sighing noise from the CPU fan is a bit trying (smile),
> & I'm considering setting up an incremental back-up system
> so that if the SSD collapses,
> I can restore everything from files stored on the HDD without re-installing.
>
> I already have an adequate set-up of back-ups, incl off-site copies,
> which ensures my personal files + configs cb restored,
> but it involves some human intervention from me at various times
> & I would have to re-install all the system + Portage stuff.
>
> What do people who do incremental back-ups use ?

I'm using rdiff-backup to do incremental backups of my local machine
to an external HDD, as well as using it to make backups of a remote
server over SSH onto my local machine.

Re: [gentoo-user] new machine: incremental back-ups

[Doug Hunley]

On Thu, Oct 4, 2012 at 11:22 AM, Paul Hartman
<paul.hartman+gentoo@gmail.com> wrote:
> I'm using rdiff-backup to do incremental backups of my local machine
> to an external HDD, as well as using it to make backups of a remote
> server over SSH onto my local machine.

Using rdiff-backup as well here on all my machines, stuffing them over
the network onto a large RAID that is then copied offsite (again with
rdiff-backup)

-- 
Douglas J Hunley (doug.hunley@gmail.com)
Twitter: @hunleyd                                               Web:
douglasjhunley.com
G+: http://goo.gl/sajR3

[gentoo-user] Re: new machine: incremental back-ups

[Grant Edwards]

On 2012-10-04, Philip Webb <purslow@ca.inter.net> wrote:
> My new machine is working very well -- thanks again for the advice -- ,
> tho' the sighing noise from the CPU fan is a bit trying (smile),
> & I'm considering setting up an incremental back-up system
> so that if the SSD collapses,
> I can restore everything from files stored on the HDD without re-installing.
>
> I already have an adequate set-up of back-ups, incl off-site copies,
> which ensures my personal files + configs cb restored,
> but it involves some human intervention from me at various times
> & I would have to re-install all the system + Portage stuff.
>
> What do people who do incremental back-ups use ?

rsnapshot

http://www.rsnapshot.org/
http://en.gentoo-wiki.com/wiki/Rsnapshot
http://www.gentoo-wiki.info/Backup#rsnapshot
http://wiki.gentoo.org/wiki/A_simple_backup_scheme_using_rsnapshot

-- 
Grant Edwards               grant.b.edwards        Yow! Does someone from
                                  at               PEORIA have a SHORTER
                              gmail.com            ATTENTION span than me?

Re: [gentoo-user] Re: new machine: incremental back-ups

[Philip Webb]

121005 Grant Edwards wrote:
> On 2012-10-04, Philip Webb <purslow@ca.inter.net> wrote:
>> & I'm considering setting up an incremental back-up system
>> so that if the SSD collapses,
>> I can restore everything from files stored on the HDD without re-installing.
>> What do people who do incremental back-ups use ?
> rsnapshot :
> http://www.rsnapshot.org/
> http://en.gentoo-wiki.com/wiki/Rsnapshot
> http://www.gentoo-wiki.info/Backup#rsnapshot
> http://wiki.gentoo.org/wiki/A_simple_backup_scheme_using_rsnapshot

Thanks for this & the other suggestions.

I had a look at Rsnapshot, but your docs make its usefulness clearer,
esp the "simple scheme".  Dirvish hasn't been updated since 2007 ;
someone appears to have mentioned Dar, but I didn't receive the msg ;
Backuppc + Rdiff-backup look possible ;
there's also Kbackup & many others under the  app-backup  dir.

A "simple scheme" of incrementally backing-up my home dir daily,
home dir + /etc + others weekly & the whole system monthly is my need;
uploading the increments off-site wb useful too, if not too large.
At present, I follow a less intensive version of this scheme,
but have to do the various jobs by hand.

The machine is running  c 16 hr/day , so I have to ensure
that cron jobs wb run at times when it's awake,
which is not predictable on any given day.
IIRC that used to require Anacron, but that's not installed now.

Further comments are welcome.  HTH others.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] Re: new machine: incremental back-ups

[Helmut Jarausch]

On 10/06/2012 02:49:08 AM, Philip Webb wrote:
> 
> The machine is running  c 16 hr/day , so I have to ensure
> that cron jobs wb run at times when it's awake,
> which is not predictable on any given day.

Have a look at fcron. You can specify an hour *after startup* of the  
machine.

Helmut.

Re: [gentoo-user] Re: new machine: incremental back-ups

[Stefan Schmiedl]

Philip Webb (05.10. 20:49):

> The machine is running  c 16 hr/day , so I have to ensure
> that cron jobs wb run at times when it's awake,
> which is not predictable on any given day.

Or, if you don't especially care when the box actually shuts down,
you could put something nifty into /etc/local.d/backup.stop.

s.