public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-user] rsync backup system
@ 2009-11-19 16:44 Grant
  2009-11-19 17:25 ` Neil Bothwick
  2009-11-19 17:58 ` Albert Hopkins
  0 siblings, 2 replies; 22+ messages in thread
From: Grant @ 2009-11-19 16:44 UTC (permalink / raw
  To: Gentoo mailing list

I just finished an rsync backup system that works like this:

Each of 4 Gentoo systems contains a folder called "backup" which
contains symlinks to local files and folders for backup.  2 of the
systems contain a folder called "sync" which contains the contents of
the "backup" folder for each of the 4 systems.  3 of the systems rsync
with one of the "sync" systems, and that system rsyncs with the other
"sync" system.

I've got a few questions for you guys about this.

1. I back up the entire /etc folder of each system, and some files
have read-only permissions.  This means I get "permission denied" when
I try to rsync them.  How would you handle this?

2. Some of the files I back up only allow root to read.  I can run
rsync as root on each system, but I don't allow root logins.  This
means in order to rsync the second "sync" system with the first "sync"
system, I must run the rsync command from the first "sync" system.
This means I have to run rsyncd on the second "sync" system in
addition to the first "sync" system.  I'd rather only run one instance
of rsyncd.  Can anyone think of another option?

3. The rsync process always completes with "rsync error: some
files/attrs were not transferred".  How can I get more information
about which files this pertains to?

4. Should I be comfortable running the entire sync operation every
night, or am I jeopardizing the longevity of my HDs?

5. If I end up with filesystem corruption on the SRC system, will that
corruption transfer over to the DST system during an rsync, or will
the transfer just fail?

6. Can I run rsyncd on a system facing the internet without fear?

- Grant



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 16:44 [gentoo-user] rsync backup system Grant
@ 2009-11-19 17:25 ` Neil Bothwick
  2009-11-19 18:19   ` Grant
  2009-11-20 16:05   ` Grant
  2009-11-19 17:58 ` Albert Hopkins
  1 sibling, 2 replies; 22+ messages in thread
From: Neil Bothwick @ 2009-11-19 17:25 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1777 bytes --]

On Thu, 19 Nov 2009 08:44:44 -0800, Grant wrote:

> 2. Some of the files I back up only allow root to read.  I can run
> rsync as root on each system, but I don't allow root logins.  This
> means in order to rsync the second "sync" system with the first "sync"
> system, I must run the rsync command from the first "sync" system.
> This means I have to run rsyncd on the second "sync" system in
> addition to the first "sync" system.  I'd rather only run one instance
> of rsyncd.  Can anyone think of another option?

Allow root logins only with a key, set up a specific user on the backup
server to run the backup tasks and add that users key to the
authorized_users file on the machine to be backed up.

> 3. The rsync process always completes with "rsync error: some
> files/attrs were not transferred".  How can I get more information
> about which files this pertains to?

Run rsync with the verbose option and direct stdout and stderr to files.

> 5. If I end up with filesystem corruption on the SRC system, will that
> corruption transfer over to the DST system during an rsync, or will
> the transfer just fail?

If data is corrupted, that will be backed up. If the filesystem
corruption causes read errors, rsync will bail out.

> 6. Can I run rsyncd on a system facing the internet without fear?

Yes, as long as the rsync ports are closed in your router.

Instead of all this, I'd recommend BackupPC. It handles all of your
issues and more and is efficient at backing up multiple machines. You
could run one BackupPC server and then rsync its store to the backup
backup server. The latest version in portage is old, get the 3.x ebuild
from bgo.


-- 
Neil Bothwick

Of all the people I've met you're certainly one of them

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 16:44 [gentoo-user] rsync backup system Grant
  2009-11-19 17:25 ` Neil Bothwick
@ 2009-11-19 17:58 ` Albert Hopkins
  2009-11-19 18:18   ` Albert Hopkins
  1 sibling, 1 reply; 22+ messages in thread
From: Albert Hopkins @ 2009-11-19 17:58 UTC (permalink / raw
  To: gentoo-user

On Thu, 2009-11-19 at 08:44 -0800, Grant wrote:
> I just finished an rsync backup system that works like this:
> 
> Each of 4 Gentoo systems contains a folder called "backup" which
> contains symlinks to local files and folders for backup.  2 of the
> systems contain a folder called "sync" which contains the contents of
> the "backup" folder for each of the 4 systems.  3 of the systems rsync
> with one of the "sync" systems, and that system rsyncs with the other
> "sync" system.
> 

> I've got a few questions for you guys about this.
> 
> 1. I back up the entire /etc folder of each system, and some files
> have read-only permissions.  This means I get "permission denied" when
> I try to rsync them.  How would you handle this?

I've never had this problem.  You should only need read permissions to
copy a file:

        $ touch this
        $ chmod 0400 this
        $ rsync -a this that
        $ /bin/cp -f /etc/issue this
        $ chmod 0400 this
        $ rsync -a this that
        $ ls -l this that
        4.0K -r-------- 1 percy users 3 Nov 19 12:38 that
        4.0K -r-------- 1 percy users 3 Nov 19 12:38 this


> 2. Some of the files I back up only allow root to read.  I can run
> rsync as root on each system, but I don't allow root logins.  This
> means in order to rsync the second "sync" system with the first "sync"
> system, I must run the rsync command from the first "sync" system.
> This means I have to run rsyncd on the second "sync" system in
> addition to the first "sync" system.  I'd rather only run one instance
> of rsyncd.  Can anyone think of another option?
> 
Well if #1 was not run as root then this is where you have problems.
And you will continue to have problems if you don't copy as root or some
user who has at least read access to all the files on each source and
can also preserve ownership/permissions on the target system.  Like...
root.

ssh.  Why are you using rsyncd anyway?  AFAIK rsyncd is not encrypted.
You can allow a non-root user to ssh in and run rsync as root via a
proxy command. You can also configure sshd to only allow root to log in
and execute a particular command (such as rsync).  See PermitRootLogin
and ForcedCommand under ssh_config(5)

> 3. The rsync process always completes with "rsync error: some
> files/attrs were not transferred".  How can I get more information
> about which files this pertains to?

It should spit it out to stderr.  If you using -v they probably scrolled
by.  Redirect stderr to a file. If you are running rsync as non-root
then it can't preserve ownership and some perms.  If you are using
extended attributes/ACLs and did not tell rsync to use them (or comple
support in) or if the target system does not support them then they will
not be transferred. 

> 
> 4. Should I be comfortable running the entire sync operation every
> night, or am I jeopardizing the longevity of my HDs?
> 
This is a joke.

> 5. If I end up with filesystem corruption on the SRC system, will that
> corruption transfer over to the DST system during an rsync, or will
> the transfer just fail?
> 
If the filesystem returns an error to rsync then rsync will error out.
Simple as that.  If the file is currupt (not the filesystem) then,
standard garbage-in/garbage-out rules apply.

> 6. Can I run rsyncd on a system facing the internet without fear?

No. Rsyncd is not encrypted (see above) also the authentication is weak
(see the man page).

Use ssh.  It's more secure and had better lock-down mechanisms. Better
yet, use a VPN and ssh through the VPN (double authentication (and
encryption)).





^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 17:58 ` Albert Hopkins
@ 2009-11-19 18:18   ` Albert Hopkins
  2009-11-20 16:03     ` Grant
  0 siblings, 1 reply; 22+ messages in thread
From: Albert Hopkins @ 2009-11-19 18:18 UTC (permalink / raw
  To: gentoo-user

On Thu, 2009-11-19 at 12:58 -0500, Albert Hopkins wrote:
> > 
> > 4. Should I be comfortable running the entire sync operation every
> > night, or am I jeopardizing the longevity of my HDs?
> > 
> This is a joke.

I should apologize and explain this better.

If you bought a fancy expensive hard drive then it's probably designed
for extremely heavy use and comes with nice coverage by the guy that
sold it to you.  But since you ask this question I'll assume that it's
not the case.

Ok so you bought a relatively cheap heard drive.  But the question seems
irrelevant.  Because these are cheap, mass-produced hard drives you
could do absolutely nothing with them and they could still die tomorrow.
That's the price you pay for cheaper drives.  Cheap hard drives are like
life.  Life is cheap.  You could take all kinds of precautions and still
die tomorrow.  But your drive is cheap and easily replaced (esp. if it's
still under warranty).  Your data on the other hand is not.  I'd rather
have a dead drive with the data backed up than a dead drive with no
backup.  Drives are so cheap nowadays it's a non-issue.  I'm actually
hoping my 2-year old drive dies soon because it will give me an excuse
to go out and buy a *bigger* one for the same price.   But I feel
comfortable with that because I keep backups.




^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 17:25 ` Neil Bothwick
@ 2009-11-19 18:19   ` Grant
  2009-11-20 17:05     ` Ward Poelmans
  2009-11-20 16:05   ` Grant
  1 sibling, 1 reply; 22+ messages in thread
From: Grant @ 2009-11-19 18:19 UTC (permalink / raw
  To: gentoo-user

>> 2. Some of the files I back up only allow root to read.  I can run
>> rsync as root on each system, but I don't allow root logins.  This
>> means in order to rsync the second "sync" system with the first "sync"
>> system, I must run the rsync command from the first "sync" system.
>> This means I have to run rsyncd on the second "sync" system in
>> addition to the first "sync" system.  I'd rather only run one instance
>> of rsyncd.  Can anyone think of another option?
>
> Allow root logins only with a key, set up a specific user on the backup
> server to run the backup tasks and add that users key to the
> authorized_users file on the machine to be backed up.
>
>> 3. The rsync process always completes with "rsync error: some
>> files/attrs were not transferred".  How can I get more information
>> about which files this pertains to?
>
> Run rsync with the verbose option and direct stdout and stderr to files.
>
>> 5. If I end up with filesystem corruption on the SRC system, will that
>> corruption transfer over to the DST system during an rsync, or will
>> the transfer just fail?
>
> If data is corrupted, that will be backed up. If the filesystem
> corruption causes read errors, rsync will bail out.
>
>> 6. Can I run rsyncd on a system facing the internet without fear?
>
> Yes, as long as the rsync ports are closed in your router.
>
> Instead of all this, I'd recommend BackupPC. It handles all of your
> issues and more and is efficient at backing up multiple machines. You
> could run one BackupPC server and then rsync its store to the backup
> backup server. The latest version in portage is old, get the 3.x ebuild
> from bgo.

BackupPC does look pretty good.  Would anyone recommend I *don't* can
this whole thing and set up BackupPC instead?

- Grant



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 18:18   ` Albert Hopkins
@ 2009-11-20 16:03     ` Grant
  2009-11-20 17:31       ` Albert Hopkins
  0 siblings, 1 reply; 22+ messages in thread
From: Grant @ 2009-11-20 16:03 UTC (permalink / raw
  To: gentoo-user

>> > 4. Should I be comfortable running the entire sync operation every
>> > night, or am I jeopardizing the longevity of my HDs?
>> >
>> This is a joke.
>
> I should apologize and explain this better.
>
> If you bought a fancy expensive hard drive then it's probably designed
> for extremely heavy use and comes with nice coverage by the guy that
> sold it to you.  But since you ask this question I'll assume that it's
> not the case.
>
> Ok so you bought a relatively cheap heard drive.  But the question seems
> irrelevant.  Because these are cheap, mass-produced hard drives you
> could do absolutely nothing with them and they could still die tomorrow.
> That's the price you pay for cheaper drives.  Cheap hard drives are like
> life.  Life is cheap.  You could take all kinds of precautions and still
> die tomorrow.  But your drive is cheap and easily replaced (esp. if it's
> still under warranty).  Your data on the other hand is not.  I'd rather
> have a dead drive with the data backed up than a dead drive with no
> backup.  Drives are so cheap nowadays it's a non-issue.  I'm actually
> hoping my 2-year old drive dies soon because it will give me an excuse
> to go out and buy a *bigger* one for the same price.   But I feel
> comfortable with that because I keep backups.

Thank you for the clarification.  Which are the "fancy expensive" hard drives?

- Grant



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 17:25 ` Neil Bothwick
  2009-11-19 18:19   ` Grant
@ 2009-11-20 16:05   ` Grant
  2009-11-20 22:08     ` Neil Bothwick
  1 sibling, 1 reply; 22+ messages in thread
From: Grant @ 2009-11-20 16:05 UTC (permalink / raw
  To: gentoo-user

>> 2. Some of the files I back up only allow root to read.  I can run
>> rsync as root on each system, but I don't allow root logins.  This
>> means in order to rsync the second "sync" system with the first "sync"
>> system, I must run the rsync command from the first "sync" system.
>> This means I have to run rsyncd on the second "sync" system in
>> addition to the first "sync" system.  I'd rather only run one instance
>> of rsyncd.  Can anyone think of another option?
>
> Allow root logins only with a key, set up a specific user on the backup
> server to run the backup tasks and add that users key to the
> authorized_users file on the machine to be backed up.
>
>> 3. The rsync process always completes with "rsync error: some
>> files/attrs were not transferred".  How can I get more information
>> about which files this pertains to?
>
> Run rsync with the verbose option and direct stdout and stderr to files.
>
>> 5. If I end up with filesystem corruption on the SRC system, will that
>> corruption transfer over to the DST system during an rsync, or will
>> the transfer just fail?
>
> If data is corrupted, that will be backed up. If the filesystem
> corruption causes read errors, rsync will bail out.
>
>> 6. Can I run rsyncd on a system facing the internet without fear?
>
> Yes, as long as the rsync ports are closed in your router.
>
> Instead of all this, I'd recommend BackupPC. It handles all of your
> issues and more and is efficient at backing up multiple machines. You
> could run one BackupPC server and then rsync its store to the backup
> backup server. The latest version in portage is old, get the 3.x ebuild
> from bgo.

How is BackupPC to set up?  Is it a whole new world to explore, or can
it be set up quickly and easily?

 - Grant



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-19 18:19   ` Grant
@ 2009-11-20 17:05     ` Ward Poelmans
  0 siblings, 0 replies; 22+ messages in thread
From: Ward Poelmans @ 2009-11-20 17:05 UTC (permalink / raw
  To: gentoo-user

On Thu, Nov 19, 2009 at 19:19, Grant <emailgrant@gmail.com> wrote:
> BackupPC does look pretty good.  Would anyone recommend I *don't* can
> this whole thing and set up BackupPC instead?

I recommend you take a look at rsnapshot instead of pure rsync.

Ward



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-20 16:03     ` Grant
@ 2009-11-20 17:31       ` Albert Hopkins
  0 siblings, 0 replies; 22+ messages in thread
From: Albert Hopkins @ 2009-11-20 17:31 UTC (permalink / raw
  To: gentoo-user

On Fri, 2009-11-20 at 08:03 -0800, Grant wrote:
> Thank you for the clarification.  Which are the "fancy expensive" hard
> drives?

The SAS drives that run at 15k RPM and cost $2-3 USD per GB.  As opposed
to your "run of the mill" 7200RPM SATA drive that costs pennies per GB.






^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-20 16:05   ` Grant
@ 2009-11-20 22:08     ` Neil Bothwick
  2010-02-24 19:02       ` Grant
  0 siblings, 1 reply; 22+ messages in thread
From: Neil Bothwick @ 2009-11-20 22:08 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

On Fri, 20 Nov 2009 08:05:27 -0800, Grant wrote:

> How is BackupPC to set up?  Is it a whole new world to explore, or can
> it be set up quickly and easily?

It takes a little while to get the hang of how the config files work, but
once you get it it takes no work at all. Restoring is as simple as
selecting the files you want in a browser and pressing a button.


-- 
Neil Bothwick

LISP: Lots of Infuriating & Silly Parentheses

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2009-11-20 22:08     ` Neil Bothwick
@ 2010-02-24 19:02       ` Grant
  2010-02-24 20:51         ` Neil Bothwick
  0 siblings, 1 reply; 22+ messages in thread
From: Grant @ 2010-02-24 19:02 UTC (permalink / raw
  To: gentoo-user

>> How is BackupPC to set up?  Is it a whole new world to explore, or can
>> it be set up quickly and easily?
>
> It takes a little while to get the hang of how the config files work, but
> once you get it it takes no work at all. Restoring is as simple as
> selecting the files you want in a browser and pressing a button.
>
>
> --
> Neil Bothwick

Has anyone tried backupninja?  There is a new ebuild for it.

https://labs.riseup.net/code/projects/show/backupninja/

Is BackupPC too excellent to consider an alternative?  I'm going to
set up one of these backup systems in the next few weeks.

- Grant



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-24 19:02       ` Grant
@ 2010-02-24 20:51         ` Neil Bothwick
  2010-02-25 15:15           ` Ward Poelmans
  0 siblings, 1 reply; 22+ messages in thread
From: Neil Bothwick @ 2010-02-24 20:51 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

On Wed, 24 Feb 2010 11:02:42 -0800, Grant wrote:

> Has anyone tried backupninja?  There is a new ebuild for it.
> 
> https://labs.riseup.net/code/projects/show/backupninja/
> 
> Is BackupPC too excellent to consider an alternative?  I'm going to
> set up one of these backup systems in the next few weeks.

It looks interesting, and no program is that good that alternatives
should never be considered, but I really like the way BaclupPC works.
Everything is handled by the server, all you need to do on each client is
copy the backuppc user's public key to /root/.ssh/authorized_keys.


-- 
Neil Bothwick

The dark ages were caused by the Y1K problem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-24 20:51         ` Neil Bothwick
@ 2010-02-25 15:15           ` Ward Poelmans
  2010-02-25 15:41             ` Alan McKinnon
  2010-02-25 16:06             ` Neil Bothwick
  0 siblings, 2 replies; 22+ messages in thread
From: Ward Poelmans @ 2010-02-25 15:15 UTC (permalink / raw
  To: gentoo-user

On Wed, Feb 24, 2010 at 21:51, Neil Bothwick <neil@digimed.co.uk> wrote:
>
> It looks interesting, and no program is that good that alternatives
> should never be considered, but I really like the way BaclupPC works.
> Everything is handled by the server, all you need to do on each client is
> copy the backuppc user's public key to /root/.ssh/authorized_keys.

Well, that's one of the things i don't like. Is there backup software
where the client does the backup to the server and not the server
fetching the backup from the client? I can't find a good way to take
regular backup's from laptop that come and go.

Regards,

Ward



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 15:15           ` Ward Poelmans
@ 2010-02-25 15:41             ` Alan McKinnon
  2010-02-25 16:11               ` Ward Poelmans
  2010-02-25 17:45               ` Neil Bothwick
  2010-02-25 16:06             ` Neil Bothwick
  1 sibling, 2 replies; 22+ messages in thread
From: Alan McKinnon @ 2010-02-25 15:41 UTC (permalink / raw
  To: gentoo-user

On Thursday 25 February 2010 17:15:36 Ward Poelmans wrote:
> On Wed, Feb 24, 2010 at 21:51, Neil Bothwick <neil@digimed.co.uk> wrote:
> > It looks interesting, and no program is that good that alternatives
> > should never be considered, but I really like the way BaclupPC works.
> > Everything is handled by the server, all you need to do on each client is
> > copy the backuppc user's public key to /root/.ssh/authorized_keys.
> 
> Well, that's one of the things i don't like. Is there backup software
> where the client does the backup to the server and not the server
> fetching the backup from the client? I can't find a good way to take
> regular backup's from laptop that come and go.


Never mind the massive security issues resulting from the backup server 
logging into the clients with a passphrase-less key, AS ROOT.

Which means you now have to open up root logins over ssh on the clients.

And someone gets into your backup server, BANG! instant pwnage of every single 
machine on your network. Heck, you don't even have to try and compromise the 
local root account, you already have full unfettered access to everything 
anyway. Worse, I'll bet the server software runs as an unpriviledged user, so 
you can just bypass the bit where you have to compromise root there as well.

It all looks like a classic case of sacrificing any and all security in the 
name of mere convenience. I have the same running battle at work, some idiot 
(probably in marketing getting a kickback) wants me to run a frigging JAVA 
backup app on my mission critical servers, the ones with 500+ users on them 
most of them chancers of the first order. They didn't understand the irony 
when I suggested I should just dispense with passwords and keys altogether and 
chmod -R 777 /

A much better way is to run a dedicated agent on the client. If the server 
needs to schedule backups, it can ask the agent to do so using regular tcp 
traffic. The client can then do it's backup and rsync it over to the server 
when it's done, and that push can be done as a regular user on both ends. The 
actual backing up on the client must be done by root of course, no other user 
has the necessary access.



-- 
alan dot mckinnon at gmail dot com



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 15:15           ` Ward Poelmans
  2010-02-25 15:41             ` Alan McKinnon
@ 2010-02-25 16:06             ` Neil Bothwick
  1 sibling, 0 replies; 22+ messages in thread
From: Neil Bothwick @ 2010-02-25 16:06 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 906 bytes --]

On Thu, 25 Feb 2010 16:15:36 +0100, Ward Poelmans wrote:

> > It looks interesting, and no program is that good that alternatives
> > should never be considered, but I really like the way BaclupPC works.
> > Everything is handled by the server, all you need to do on each
> > client is copy the backuppc user's public key
> > to /root/.ssh/authorized_keys.  
> 
> Well, that's one of the things i don't like. Is there backup software
> where the client does the backup to the server and not the server
> fetching the backup from the client? I can't find a good way to take
> regular backup's from laptop that come and go.

You can manually start a backup using the BackupPC web interface, but the
automatic backup thing works with laptops, the server just waits until
the laptop appears on the network.


-- 
Neil Bothwick

Synonym: a word you use when you can't spell the other one.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 15:41             ` Alan McKinnon
@ 2010-02-25 16:11               ` Ward Poelmans
  2010-02-25 17:50                 ` daid kahl
                                   ` (2 more replies)
  2010-02-25 17:45               ` Neil Bothwick
  1 sibling, 3 replies; 22+ messages in thread
From: Ward Poelmans @ 2010-02-25 16:11 UTC (permalink / raw
  To: gentoo-user

On Thu, Feb 25, 2010 at 16:41, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> A much better way is to run a dedicated agent on the client. If the server
> needs to schedule backups, it can ask the agent to do so using regular tcp
> traffic. The client can then do it's backup and rsync it over to the server
> when it's done, and that push can be done as a regular user on both ends. The
> actual backing up on the client must be done by root of course, no other user
> has the necessary access.

Sounds great. Is there any software that works this way?

Ward



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 15:41             ` Alan McKinnon
  2010-02-25 16:11               ` Ward Poelmans
@ 2010-02-25 17:45               ` Neil Bothwick
  1 sibling, 0 replies; 22+ messages in thread
From: Neil Bothwick @ 2010-02-25 17:45 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1303 bytes --]

On Thu, 25 Feb 2010 17:41:13 +0200, Alan McKinnon wrote:

> And someone gets into your backup server, BANG! instant pwnage of every
> single machine on your network. Heck, you don't even have to try and
> compromise the local root account, you already have full unfettered
> access to everything anyway.

Which is why you don't allow access to the backup server from outside of
the network, and restrict root access from inside. Because backups are
initiated from the server, it doesn't actually need any ports open to do
its job, although a web server is needed to run the user interface
(which isn't necessary). The ebuild sets up a separate instance of
Apache just for this, so even if you are already running Apache on the
backup server (which is a crazy idea to start with) compromising that
won't get you into the backups.

> Worse, I'll bet the server software runs
> as an unpriviledged user, so you can just bypass the bit where you have
> to compromise root there as well.

You lose :P

The server runs as a restricted user, with no login shell.


-- 
Neil Bothwick

WinErr 042: Virus error - A virus has been activated in a dos-box. The
virus, however, requires Windows. All tasks will automatically be closed
            and the virus will be activated again.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 16:11               ` Ward Poelmans
@ 2010-02-25 17:50                 ` daid kahl
  2010-02-26 13:23                   ` Ward Poelmans
  2010-02-25 17:52                 ` Grant
  2010-03-17 15:33                 ` Ward Poelmans
  2 siblings, 1 reply; 22+ messages in thread
From: daid kahl @ 2010-02-25 17:50 UTC (permalink / raw
  To: gentoo-user

On 26 February 2010 01:11, Ward Poelmans <wpoely86@gmail.com> wrote:
> On Thu, Feb 25, 2010 at 16:41, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> A much better way is to run a dedicated agent on the client. If the server
>> needs to schedule backups, it can ask the agent to do so using regular tcp
>> traffic. The client can then do it's backup and rsync it over to the server
>> when it's done, and that push can be done as a regular user on both ends. The
>> actual backing up on the client must be done by root of course, no other user
>> has the necessary access.
>
> Sounds great. Is there any software that works this way?
>
> Ward

Sounds more or less like cron tasks and rsnapshot to me (can use other
rsync scripts of course, but this one is nice to me anyway, and
someone else mentioned it earlier in the thread).  I'm not sure off
hand I have a good way for it to be initialized from the server end,
but if it's a backup, it might as well run on a local cron anyway
rather than needing an external call.

As a simple idea, cron task starts rsnapshot configured however.  When
this is done, backup is tarballed, and tarball is given as like, say,
440 permissions, where users are in some useful 'backup' group, then
while tarball can be read to be passed across server, if tarball is
extracted, user has no more privs then they have on the system anyway
(I'm not saying chmod -R).  Then local tarball can be removed or
whatever.

And call me silly for not reading documentation or assuming, but I was
very happy last night when I realized system rescue CD includes
rsnapshot already!

~daid



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 16:11               ` Ward Poelmans
  2010-02-25 17:50                 ` daid kahl
@ 2010-02-25 17:52                 ` Grant
  2010-03-17 15:33                 ` Ward Poelmans
  2 siblings, 0 replies; 22+ messages in thread
From: Grant @ 2010-02-25 17:52 UTC (permalink / raw
  To: gentoo-user

>> A much better way is to run a dedicated agent on the client. If the server
>> needs to schedule backups, it can ask the agent to do so using regular tcp
>> traffic. The client can then do it's backup and rsync it over to the server
>> when it's done, and that push can be done as a regular user on both ends. The
>> actual backing up on the client must be done by root of course, no other user
>> has the necessary access.
>
> Sounds great. Is there any software that works this way?
>
> Ward

I'd like to know too.  :)

- Grant



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 17:50                 ` daid kahl
@ 2010-02-26 13:23                   ` Ward Poelmans
  2010-02-26 17:33                     ` daid kahl
  0 siblings, 1 reply; 22+ messages in thread
From: Ward Poelmans @ 2010-02-26 13:23 UTC (permalink / raw
  To: gentoo-user

On Thu, Feb 25, 2010 at 18:50, daid kahl <daidxor@gmail.com> wrote:

> As a simple idea, cron task starts rsnapshot configured however.  When
> this is done, backup is tarballed, and tarball is given as like, say,
> 440 permissions, where users are in some useful 'backup' group, then
> while tarball can be read to be passed across server, if tarball is
> extracted, user has no more privs then they have on the system anyway
> (I'm not saying chmod -R).  Then local tarball can be removed or
> whatever.

It's not a bad idea, but you need enough free space on the client to
backup the entire system (which for me is not the case). Secondly,
every backup you do is a full backup as rsnapshot needs to access a
backup todo a incremental backup. You could mess around with something
like sshfs but's it's not great either. A straight rsync between
client and server could do it but it would suprise me if this doesn't
already exist in some form.

Regards,

Ward



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-26 13:23                   ` Ward Poelmans
@ 2010-02-26 17:33                     ` daid kahl
  0 siblings, 0 replies; 22+ messages in thread
From: daid kahl @ 2010-02-26 17:33 UTC (permalink / raw
  To: gentoo-user

On 26 February 2010 22:23, Ward Poelmans <wpoely86@gmail.com> wrote:
> On Thu, Feb 25, 2010 at 18:50, daid kahl <daidxor@gmail.com> wrote:
>
>> As a simple idea, cron task starts rsnapshot configured however.  When
>> this is done, backup is tarballed, and tarball is given as like, say,
>> 440 permissions, where users are in some useful 'backup' group, then
>> while tarball can be read to be passed across server, if tarball is
>> extracted, user has no more privs then they have on the system anyway
>> (I'm not saying chmod -R).  Then local tarball can be removed or
>> whatever.
>
> It's not a bad idea, but you need enough free space on the client to
> backup the entire system (which for me is not the case). Secondly,
> every backup you do is a full backup as rsnapshot needs to access a
> backup todo a incremental backup. You could mess around with something
> like sshfs but's it's not great either. A straight rsync between
> client and server could do it but it would suprise me if this doesn't
> already exist in some form.
>
> Regards,
>
> Ward
>
Thanks for the feedback.  For now, as you may easily guess, this case
does not apply to me personally since I mostly just admin my own
personal machine.  But I think you raise very relevant difficulties
with my suggestion for a practical administrative case for multiple
machines.

~daid



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [gentoo-user] rsync backup system
  2010-02-25 16:11               ` Ward Poelmans
  2010-02-25 17:50                 ` daid kahl
  2010-02-25 17:52                 ` Grant
@ 2010-03-17 15:33                 ` Ward Poelmans
  2 siblings, 0 replies; 22+ messages in thread
From: Ward Poelmans @ 2010-03-17 15:33 UTC (permalink / raw
  To: gentoo-user

On Thu, Feb 25, 2010 at 17:11, Ward Poelmans <wpoely86@gmail.com> wrote:
> On Thu, Feb 25, 2010 at 16:41, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> A much better way is to run a dedicated agent on the client. If the server
>> needs to schedule backups, it can ask the agent to do so using regular tcp
>> traffic. The client can then do it's backup and rsync it over to the server
>> when it's done, and that push can be done as a regular user on both ends. The
>> actual backing up on the client must be done by root of course, no other user
>> has the necessary access.

If anyone is still interested, i had some time and this is what i did:

On the client:
rsync -a -X -b --backup-dir=../backup.0/ --link-dest=../backup.0/
/home/ward backupserver:Backup-Laptop/backup.cur/
ssh backupserver /home/ward/shiftbackups.sh

This makes a directory backup.cur on the backupserver with a full
backup in it, but it's exactly only a incremental backup as it
hardlinks from backup.0 (the previous backup). The script
shiftbackups.sh moves backup.0 to backup.1 and backup.cur to backup.0
and so on...

This does more or less exactly what i wanted.

Regards,

Ward



^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2010-03-17 15:34 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-11-19 16:44 [gentoo-user] rsync backup system Grant
2009-11-19 17:25 ` Neil Bothwick
2009-11-19 18:19   ` Grant
2009-11-20 17:05     ` Ward Poelmans
2009-11-20 16:05   ` Grant
2009-11-20 22:08     ` Neil Bothwick
2010-02-24 19:02       ` Grant
2010-02-24 20:51         ` Neil Bothwick
2010-02-25 15:15           ` Ward Poelmans
2010-02-25 15:41             ` Alan McKinnon
2010-02-25 16:11               ` Ward Poelmans
2010-02-25 17:50                 ` daid kahl
2010-02-26 13:23                   ` Ward Poelmans
2010-02-26 17:33                     ` daid kahl
2010-02-25 17:52                 ` Grant
2010-03-17 15:33                 ` Ward Poelmans
2010-02-25 17:45               ` Neil Bothwick
2010-02-25 16:06             ` Neil Bothwick
2009-11-19 17:58 ` Albert Hopkins
2009-11-19 18:18   ` Albert Hopkins
2009-11-20 16:03     ` Grant
2009-11-20 17:31       ` Albert Hopkins

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox