* [gentoo-user] rsync backup system @ 2009-11-19 16:44 Grant 2009-11-19 17:25 ` Neil Bothwick 2009-11-19 17:58 ` Albert Hopkins 0 siblings, 2 replies; 22+ messages in thread From: Grant @ 2009-11-19 16:44 UTC (permalink / raw To: Gentoo mailing list I just finished an rsync backup system that works like this: Each of 4 Gentoo systems contains a folder called "backup" which contains symlinks to local files and folders for backup. 2 of the systems contain a folder called "sync" which contains the contents of the "backup" folder for each of the 4 systems. 3 of the systems rsync with one of the "sync" systems, and that system rsyncs with the other "sync" system. I've got a few questions for you guys about this. 1. I back up the entire /etc folder of each system, and some files have read-only permissions. This means I get "permission denied" when I try to rsync them. How would you handle this? 2. Some of the files I back up only allow root to read. I can run rsync as root on each system, but I don't allow root logins. This means in order to rsync the second "sync" system with the first "sync" system, I must run the rsync command from the first "sync" system. This means I have to run rsyncd on the second "sync" system in addition to the first "sync" system. I'd rather only run one instance of rsyncd. Can anyone think of another option? 3. The rsync process always completes with "rsync error: some files/attrs were not transferred". How can I get more information about which files this pertains to? 4. Should I be comfortable running the entire sync operation every night, or am I jeopardizing the longevity of my HDs? 5. If I end up with filesystem corruption on the SRC system, will that corruption transfer over to the DST system during an rsync, or will the transfer just fail? 6. Can I run rsyncd on a system facing the internet without fear? - Grant ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 16:44 [gentoo-user] rsync backup system Grant @ 2009-11-19 17:25 ` Neil Bothwick 2009-11-19 18:19 ` Grant 2009-11-20 16:05 ` Grant 2009-11-19 17:58 ` Albert Hopkins 1 sibling, 2 replies; 22+ messages in thread From: Neil Bothwick @ 2009-11-19 17:25 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 1777 bytes --] On Thu, 19 Nov 2009 08:44:44 -0800, Grant wrote: > 2. Some of the files I back up only allow root to read. I can run > rsync as root on each system, but I don't allow root logins. This > means in order to rsync the second "sync" system with the first "sync" > system, I must run the rsync command from the first "sync" system. > This means I have to run rsyncd on the second "sync" system in > addition to the first "sync" system. I'd rather only run one instance > of rsyncd. Can anyone think of another option? Allow root logins only with a key, set up a specific user on the backup server to run the backup tasks and add that users key to the authorized_users file on the machine to be backed up. > 3. The rsync process always completes with "rsync error: some > files/attrs were not transferred". How can I get more information > about which files this pertains to? Run rsync with the verbose option and direct stdout and stderr to files. > 5. If I end up with filesystem corruption on the SRC system, will that > corruption transfer over to the DST system during an rsync, or will > the transfer just fail? If data is corrupted, that will be backed up. If the filesystem corruption causes read errors, rsync will bail out. > 6. Can I run rsyncd on a system facing the internet without fear? Yes, as long as the rsync ports are closed in your router. Instead of all this, I'd recommend BackupPC. It handles all of your issues and more and is efficient at backing up multiple machines. You could run one BackupPC server and then rsync its store to the backup backup server. The latest version in portage is old, get the 3.x ebuild from bgo. -- Neil Bothwick Of all the people I've met you're certainly one of them [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 17:25 ` Neil Bothwick @ 2009-11-19 18:19 ` Grant 2009-11-20 17:05 ` Ward Poelmans 2009-11-20 16:05 ` Grant 1 sibling, 1 reply; 22+ messages in thread From: Grant @ 2009-11-19 18:19 UTC (permalink / raw To: gentoo-user >> 2. Some of the files I back up only allow root to read. I can run >> rsync as root on each system, but I don't allow root logins. This >> means in order to rsync the second "sync" system with the first "sync" >> system, I must run the rsync command from the first "sync" system. >> This means I have to run rsyncd on the second "sync" system in >> addition to the first "sync" system. I'd rather only run one instance >> of rsyncd. Can anyone think of another option? > > Allow root logins only with a key, set up a specific user on the backup > server to run the backup tasks and add that users key to the > authorized_users file on the machine to be backed up. > >> 3. The rsync process always completes with "rsync error: some >> files/attrs were not transferred". How can I get more information >> about which files this pertains to? > > Run rsync with the verbose option and direct stdout and stderr to files. > >> 5. If I end up with filesystem corruption on the SRC system, will that >> corruption transfer over to the DST system during an rsync, or will >> the transfer just fail? > > If data is corrupted, that will be backed up. If the filesystem > corruption causes read errors, rsync will bail out. > >> 6. Can I run rsyncd on a system facing the internet without fear? > > Yes, as long as the rsync ports are closed in your router. > > Instead of all this, I'd recommend BackupPC. It handles all of your > issues and more and is efficient at backing up multiple machines. You > could run one BackupPC server and then rsync its store to the backup > backup server. The latest version in portage is old, get the 3.x ebuild > from bgo. BackupPC does look pretty good. Would anyone recommend I *don't* can this whole thing and set up BackupPC instead? - Grant ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 18:19 ` Grant @ 2009-11-20 17:05 ` Ward Poelmans 0 siblings, 0 replies; 22+ messages in thread From: Ward Poelmans @ 2009-11-20 17:05 UTC (permalink / raw To: gentoo-user On Thu, Nov 19, 2009 at 19:19, Grant <emailgrant@gmail.com> wrote: > BackupPC does look pretty good. Would anyone recommend I *don't* can > this whole thing and set up BackupPC instead? I recommend you take a look at rsnapshot instead of pure rsync. Ward ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 17:25 ` Neil Bothwick 2009-11-19 18:19 ` Grant @ 2009-11-20 16:05 ` Grant 2009-11-20 22:08 ` Neil Bothwick 1 sibling, 1 reply; 22+ messages in thread From: Grant @ 2009-11-20 16:05 UTC (permalink / raw To: gentoo-user >> 2. Some of the files I back up only allow root to read. I can run >> rsync as root on each system, but I don't allow root logins. This >> means in order to rsync the second "sync" system with the first "sync" >> system, I must run the rsync command from the first "sync" system. >> This means I have to run rsyncd on the second "sync" system in >> addition to the first "sync" system. I'd rather only run one instance >> of rsyncd. Can anyone think of another option? > > Allow root logins only with a key, set up a specific user on the backup > server to run the backup tasks and add that users key to the > authorized_users file on the machine to be backed up. > >> 3. The rsync process always completes with "rsync error: some >> files/attrs were not transferred". How can I get more information >> about which files this pertains to? > > Run rsync with the verbose option and direct stdout and stderr to files. > >> 5. If I end up with filesystem corruption on the SRC system, will that >> corruption transfer over to the DST system during an rsync, or will >> the transfer just fail? > > If data is corrupted, that will be backed up. If the filesystem > corruption causes read errors, rsync will bail out. > >> 6. Can I run rsyncd on a system facing the internet without fear? > > Yes, as long as the rsync ports are closed in your router. > > Instead of all this, I'd recommend BackupPC. It handles all of your > issues and more and is efficient at backing up multiple machines. You > could run one BackupPC server and then rsync its store to the backup > backup server. The latest version in portage is old, get the 3.x ebuild > from bgo. How is BackupPC to set up? Is it a whole new world to explore, or can it be set up quickly and easily? - Grant ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-20 16:05 ` Grant @ 2009-11-20 22:08 ` Neil Bothwick 2010-02-24 19:02 ` Grant 0 siblings, 1 reply; 22+ messages in thread From: Neil Bothwick @ 2009-11-20 22:08 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 446 bytes --] On Fri, 20 Nov 2009 08:05:27 -0800, Grant wrote: > How is BackupPC to set up? Is it a whole new world to explore, or can > it be set up quickly and easily? It takes a little while to get the hang of how the config files work, but once you get it it takes no work at all. Restoring is as simple as selecting the files you want in a browser and pressing a button. -- Neil Bothwick LISP: Lots of Infuriating & Silly Parentheses [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-20 22:08 ` Neil Bothwick @ 2010-02-24 19:02 ` Grant 2010-02-24 20:51 ` Neil Bothwick 0 siblings, 1 reply; 22+ messages in thread From: Grant @ 2010-02-24 19:02 UTC (permalink / raw To: gentoo-user >> How is BackupPC to set up? Is it a whole new world to explore, or can >> it be set up quickly and easily? > > It takes a little while to get the hang of how the config files work, but > once you get it it takes no work at all. Restoring is as simple as > selecting the files you want in a browser and pressing a button. > > > -- > Neil Bothwick Has anyone tried backupninja? There is a new ebuild for it. https://labs.riseup.net/code/projects/show/backupninja/ Is BackupPC too excellent to consider an alternative? I'm going to set up one of these backup systems in the next few weeks. - Grant ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-24 19:02 ` Grant @ 2010-02-24 20:51 ` Neil Bothwick 2010-02-25 15:15 ` Ward Poelmans 0 siblings, 1 reply; 22+ messages in thread From: Neil Bothwick @ 2010-02-24 20:51 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 673 bytes --] On Wed, 24 Feb 2010 11:02:42 -0800, Grant wrote: > Has anyone tried backupninja? There is a new ebuild for it. > > https://labs.riseup.net/code/projects/show/backupninja/ > > Is BackupPC too excellent to consider an alternative? I'm going to > set up one of these backup systems in the next few weeks. It looks interesting, and no program is that good that alternatives should never be considered, but I really like the way BaclupPC works. Everything is handled by the server, all you need to do on each client is copy the backuppc user's public key to /root/.ssh/authorized_keys. -- Neil Bothwick The dark ages were caused by the Y1K problem. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-24 20:51 ` Neil Bothwick @ 2010-02-25 15:15 ` Ward Poelmans 2010-02-25 15:41 ` Alan McKinnon 2010-02-25 16:06 ` Neil Bothwick 0 siblings, 2 replies; 22+ messages in thread From: Ward Poelmans @ 2010-02-25 15:15 UTC (permalink / raw To: gentoo-user On Wed, Feb 24, 2010 at 21:51, Neil Bothwick <neil@digimed.co.uk> wrote: > > It looks interesting, and no program is that good that alternatives > should never be considered, but I really like the way BaclupPC works. > Everything is handled by the server, all you need to do on each client is > copy the backuppc user's public key to /root/.ssh/authorized_keys. Well, that's one of the things i don't like. Is there backup software where the client does the backup to the server and not the server fetching the backup from the client? I can't find a good way to take regular backup's from laptop that come and go. Regards, Ward ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 15:15 ` Ward Poelmans @ 2010-02-25 15:41 ` Alan McKinnon 2010-02-25 16:11 ` Ward Poelmans 2010-02-25 17:45 ` Neil Bothwick 2010-02-25 16:06 ` Neil Bothwick 1 sibling, 2 replies; 22+ messages in thread From: Alan McKinnon @ 2010-02-25 15:41 UTC (permalink / raw To: gentoo-user On Thursday 25 February 2010 17:15:36 Ward Poelmans wrote: > On Wed, Feb 24, 2010 at 21:51, Neil Bothwick <neil@digimed.co.uk> wrote: > > It looks interesting, and no program is that good that alternatives > > should never be considered, but I really like the way BaclupPC works. > > Everything is handled by the server, all you need to do on each client is > > copy the backuppc user's public key to /root/.ssh/authorized_keys. > > Well, that's one of the things i don't like. Is there backup software > where the client does the backup to the server and not the server > fetching the backup from the client? I can't find a good way to take > regular backup's from laptop that come and go. Never mind the massive security issues resulting from the backup server logging into the clients with a passphrase-less key, AS ROOT. Which means you now have to open up root logins over ssh on the clients. And someone gets into your backup server, BANG! instant pwnage of every single machine on your network. Heck, you don't even have to try and compromise the local root account, you already have full unfettered access to everything anyway. Worse, I'll bet the server software runs as an unpriviledged user, so you can just bypass the bit where you have to compromise root there as well. It all looks like a classic case of sacrificing any and all security in the name of mere convenience. I have the same running battle at work, some idiot (probably in marketing getting a kickback) wants me to run a frigging JAVA backup app on my mission critical servers, the ones with 500+ users on them most of them chancers of the first order. They didn't understand the irony when I suggested I should just dispense with passwords and keys altogether and chmod -R 777 / A much better way is to run a dedicated agent on the client. If the server needs to schedule backups, it can ask the agent to do so using regular tcp traffic. The client can then do it's backup and rsync it over to the server when it's done, and that push can be done as a regular user on both ends. The actual backing up on the client must be done by root of course, no other user has the necessary access. -- alan dot mckinnon at gmail dot com ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 15:41 ` Alan McKinnon @ 2010-02-25 16:11 ` Ward Poelmans 2010-02-25 17:50 ` daid kahl ` (2 more replies) 2010-02-25 17:45 ` Neil Bothwick 1 sibling, 3 replies; 22+ messages in thread From: Ward Poelmans @ 2010-02-25 16:11 UTC (permalink / raw To: gentoo-user On Thu, Feb 25, 2010 at 16:41, Alan McKinnon <alan.mckinnon@gmail.com> wrote: > A much better way is to run a dedicated agent on the client. If the server > needs to schedule backups, it can ask the agent to do so using regular tcp > traffic. The client can then do it's backup and rsync it over to the server > when it's done, and that push can be done as a regular user on both ends. The > actual backing up on the client must be done by root of course, no other user > has the necessary access. Sounds great. Is there any software that works this way? Ward ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 16:11 ` Ward Poelmans @ 2010-02-25 17:50 ` daid kahl 2010-02-26 13:23 ` Ward Poelmans 2010-02-25 17:52 ` Grant 2010-03-17 15:33 ` Ward Poelmans 2 siblings, 1 reply; 22+ messages in thread From: daid kahl @ 2010-02-25 17:50 UTC (permalink / raw To: gentoo-user On 26 February 2010 01:11, Ward Poelmans <wpoely86@gmail.com> wrote: > On Thu, Feb 25, 2010 at 16:41, Alan McKinnon <alan.mckinnon@gmail.com> wrote: >> A much better way is to run a dedicated agent on the client. If the server >> needs to schedule backups, it can ask the agent to do so using regular tcp >> traffic. The client can then do it's backup and rsync it over to the server >> when it's done, and that push can be done as a regular user on both ends. The >> actual backing up on the client must be done by root of course, no other user >> has the necessary access. > > Sounds great. Is there any software that works this way? > > Ward Sounds more or less like cron tasks and rsnapshot to me (can use other rsync scripts of course, but this one is nice to me anyway, and someone else mentioned it earlier in the thread). I'm not sure off hand I have a good way for it to be initialized from the server end, but if it's a backup, it might as well run on a local cron anyway rather than needing an external call. As a simple idea, cron task starts rsnapshot configured however. When this is done, backup is tarballed, and tarball is given as like, say, 440 permissions, where users are in some useful 'backup' group, then while tarball can be read to be passed across server, if tarball is extracted, user has no more privs then they have on the system anyway (I'm not saying chmod -R). Then local tarball can be removed or whatever. And call me silly for not reading documentation or assuming, but I was very happy last night when I realized system rescue CD includes rsnapshot already! ~daid ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 17:50 ` daid kahl @ 2010-02-26 13:23 ` Ward Poelmans 2010-02-26 17:33 ` daid kahl 0 siblings, 1 reply; 22+ messages in thread From: Ward Poelmans @ 2010-02-26 13:23 UTC (permalink / raw To: gentoo-user On Thu, Feb 25, 2010 at 18:50, daid kahl <daidxor@gmail.com> wrote: > As a simple idea, cron task starts rsnapshot configured however. When > this is done, backup is tarballed, and tarball is given as like, say, > 440 permissions, where users are in some useful 'backup' group, then > while tarball can be read to be passed across server, if tarball is > extracted, user has no more privs then they have on the system anyway > (I'm not saying chmod -R). Then local tarball can be removed or > whatever. It's not a bad idea, but you need enough free space on the client to backup the entire system (which for me is not the case). Secondly, every backup you do is a full backup as rsnapshot needs to access a backup todo a incremental backup. You could mess around with something like sshfs but's it's not great either. A straight rsync between client and server could do it but it would suprise me if this doesn't already exist in some form. Regards, Ward ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-26 13:23 ` Ward Poelmans @ 2010-02-26 17:33 ` daid kahl 0 siblings, 0 replies; 22+ messages in thread From: daid kahl @ 2010-02-26 17:33 UTC (permalink / raw To: gentoo-user On 26 February 2010 22:23, Ward Poelmans <wpoely86@gmail.com> wrote: > On Thu, Feb 25, 2010 at 18:50, daid kahl <daidxor@gmail.com> wrote: > >> As a simple idea, cron task starts rsnapshot configured however. When >> this is done, backup is tarballed, and tarball is given as like, say, >> 440 permissions, where users are in some useful 'backup' group, then >> while tarball can be read to be passed across server, if tarball is >> extracted, user has no more privs then they have on the system anyway >> (I'm not saying chmod -R). Then local tarball can be removed or >> whatever. > > It's not a bad idea, but you need enough free space on the client to > backup the entire system (which for me is not the case). Secondly, > every backup you do is a full backup as rsnapshot needs to access a > backup todo a incremental backup. You could mess around with something > like sshfs but's it's not great either. A straight rsync between > client and server could do it but it would suprise me if this doesn't > already exist in some form. > > Regards, > > Ward > Thanks for the feedback. For now, as you may easily guess, this case does not apply to me personally since I mostly just admin my own personal machine. But I think you raise very relevant difficulties with my suggestion for a practical administrative case for multiple machines. ~daid ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 16:11 ` Ward Poelmans 2010-02-25 17:50 ` daid kahl @ 2010-02-25 17:52 ` Grant 2010-03-17 15:33 ` Ward Poelmans 2 siblings, 0 replies; 22+ messages in thread From: Grant @ 2010-02-25 17:52 UTC (permalink / raw To: gentoo-user >> A much better way is to run a dedicated agent on the client. If the server >> needs to schedule backups, it can ask the agent to do so using regular tcp >> traffic. The client can then do it's backup and rsync it over to the server >> when it's done, and that push can be done as a regular user on both ends. The >> actual backing up on the client must be done by root of course, no other user >> has the necessary access. > > Sounds great. Is there any software that works this way? > > Ward I'd like to know too. :) - Grant ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 16:11 ` Ward Poelmans 2010-02-25 17:50 ` daid kahl 2010-02-25 17:52 ` Grant @ 2010-03-17 15:33 ` Ward Poelmans 2 siblings, 0 replies; 22+ messages in thread From: Ward Poelmans @ 2010-03-17 15:33 UTC (permalink / raw To: gentoo-user On Thu, Feb 25, 2010 at 17:11, Ward Poelmans <wpoely86@gmail.com> wrote: > On Thu, Feb 25, 2010 at 16:41, Alan McKinnon <alan.mckinnon@gmail.com> wrote: >> A much better way is to run a dedicated agent on the client. If the server >> needs to schedule backups, it can ask the agent to do so using regular tcp >> traffic. The client can then do it's backup and rsync it over to the server >> when it's done, and that push can be done as a regular user on both ends. The >> actual backing up on the client must be done by root of course, no other user >> has the necessary access. If anyone is still interested, i had some time and this is what i did: On the client: rsync -a -X -b --backup-dir=../backup.0/ --link-dest=../backup.0/ /home/ward backupserver:Backup-Laptop/backup.cur/ ssh backupserver /home/ward/shiftbackups.sh This makes a directory backup.cur on the backupserver with a full backup in it, but it's exactly only a incremental backup as it hardlinks from backup.0 (the previous backup). The script shiftbackups.sh moves backup.0 to backup.1 and backup.cur to backup.0 and so on... This does more or less exactly what i wanted. Regards, Ward ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 15:41 ` Alan McKinnon 2010-02-25 16:11 ` Ward Poelmans @ 2010-02-25 17:45 ` Neil Bothwick 1 sibling, 0 replies; 22+ messages in thread From: Neil Bothwick @ 2010-02-25 17:45 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 1303 bytes --] On Thu, 25 Feb 2010 17:41:13 +0200, Alan McKinnon wrote: > And someone gets into your backup server, BANG! instant pwnage of every > single machine on your network. Heck, you don't even have to try and > compromise the local root account, you already have full unfettered > access to everything anyway. Which is why you don't allow access to the backup server from outside of the network, and restrict root access from inside. Because backups are initiated from the server, it doesn't actually need any ports open to do its job, although a web server is needed to run the user interface (which isn't necessary). The ebuild sets up a separate instance of Apache just for this, so even if you are already running Apache on the backup server (which is a crazy idea to start with) compromising that won't get you into the backups. > Worse, I'll bet the server software runs > as an unpriviledged user, so you can just bypass the bit where you have > to compromise root there as well. You lose :P The server runs as a restricted user, with no login shell. -- Neil Bothwick WinErr 042: Virus error - A virus has been activated in a dos-box. The virus, however, requires Windows. All tasks will automatically be closed and the virus will be activated again. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2010-02-25 15:15 ` Ward Poelmans 2010-02-25 15:41 ` Alan McKinnon @ 2010-02-25 16:06 ` Neil Bothwick 1 sibling, 0 replies; 22+ messages in thread From: Neil Bothwick @ 2010-02-25 16:06 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 906 bytes --] On Thu, 25 Feb 2010 16:15:36 +0100, Ward Poelmans wrote: > > It looks interesting, and no program is that good that alternatives > > should never be considered, but I really like the way BaclupPC works. > > Everything is handled by the server, all you need to do on each > > client is copy the backuppc user's public key > > to /root/.ssh/authorized_keys. > > Well, that's one of the things i don't like. Is there backup software > where the client does the backup to the server and not the server > fetching the backup from the client? I can't find a good way to take > regular backup's from laptop that come and go. You can manually start a backup using the BackupPC web interface, but the automatic backup thing works with laptops, the server just waits until the laptop appears on the network. -- Neil Bothwick Synonym: a word you use when you can't spell the other one. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 16:44 [gentoo-user] rsync backup system Grant 2009-11-19 17:25 ` Neil Bothwick @ 2009-11-19 17:58 ` Albert Hopkins 2009-11-19 18:18 ` Albert Hopkins 1 sibling, 1 reply; 22+ messages in thread From: Albert Hopkins @ 2009-11-19 17:58 UTC (permalink / raw To: gentoo-user On Thu, 2009-11-19 at 08:44 -0800, Grant wrote: > I just finished an rsync backup system that works like this: > > Each of 4 Gentoo systems contains a folder called "backup" which > contains symlinks to local files and folders for backup. 2 of the > systems contain a folder called "sync" which contains the contents of > the "backup" folder for each of the 4 systems. 3 of the systems rsync > with one of the "sync" systems, and that system rsyncs with the other > "sync" system. > > I've got a few questions for you guys about this. > > 1. I back up the entire /etc folder of each system, and some files > have read-only permissions. This means I get "permission denied" when > I try to rsync them. How would you handle this? I've never had this problem. You should only need read permissions to copy a file: $ touch this $ chmod 0400 this $ rsync -a this that $ /bin/cp -f /etc/issue this $ chmod 0400 this $ rsync -a this that $ ls -l this that 4.0K -r-------- 1 percy users 3 Nov 19 12:38 that 4.0K -r-------- 1 percy users 3 Nov 19 12:38 this > 2. Some of the files I back up only allow root to read. I can run > rsync as root on each system, but I don't allow root logins. This > means in order to rsync the second "sync" system with the first "sync" > system, I must run the rsync command from the first "sync" system. > This means I have to run rsyncd on the second "sync" system in > addition to the first "sync" system. I'd rather only run one instance > of rsyncd. Can anyone think of another option? > Well if #1 was not run as root then this is where you have problems. And you will continue to have problems if you don't copy as root or some user who has at least read access to all the files on each source and can also preserve ownership/permissions on the target system. Like... root. ssh. Why are you using rsyncd anyway? AFAIK rsyncd is not encrypted. You can allow a non-root user to ssh in and run rsync as root via a proxy command. You can also configure sshd to only allow root to log in and execute a particular command (such as rsync). See PermitRootLogin and ForcedCommand under ssh_config(5) > 3. The rsync process always completes with "rsync error: some > files/attrs were not transferred". How can I get more information > about which files this pertains to? It should spit it out to stderr. If you using -v they probably scrolled by. Redirect stderr to a file. If you are running rsync as non-root then it can't preserve ownership and some perms. If you are using extended attributes/ACLs and did not tell rsync to use them (or comple support in) or if the target system does not support them then they will not be transferred. > > 4. Should I be comfortable running the entire sync operation every > night, or am I jeopardizing the longevity of my HDs? > This is a joke. > 5. If I end up with filesystem corruption on the SRC system, will that > corruption transfer over to the DST system during an rsync, or will > the transfer just fail? > If the filesystem returns an error to rsync then rsync will error out. Simple as that. If the file is currupt (not the filesystem) then, standard garbage-in/garbage-out rules apply. > 6. Can I run rsyncd on a system facing the internet without fear? No. Rsyncd is not encrypted (see above) also the authentication is weak (see the man page). Use ssh. It's more secure and had better lock-down mechanisms. Better yet, use a VPN and ssh through the VPN (double authentication (and encryption)). ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 17:58 ` Albert Hopkins @ 2009-11-19 18:18 ` Albert Hopkins 2009-11-20 16:03 ` Grant 0 siblings, 1 reply; 22+ messages in thread From: Albert Hopkins @ 2009-11-19 18:18 UTC (permalink / raw To: gentoo-user On Thu, 2009-11-19 at 12:58 -0500, Albert Hopkins wrote: > > > > 4. Should I be comfortable running the entire sync operation every > > night, or am I jeopardizing the longevity of my HDs? > > > This is a joke. I should apologize and explain this better. If you bought a fancy expensive hard drive then it's probably designed for extremely heavy use and comes with nice coverage by the guy that sold it to you. But since you ask this question I'll assume that it's not the case. Ok so you bought a relatively cheap heard drive. But the question seems irrelevant. Because these are cheap, mass-produced hard drives you could do absolutely nothing with them and they could still die tomorrow. That's the price you pay for cheaper drives. Cheap hard drives are like life. Life is cheap. You could take all kinds of precautions and still die tomorrow. But your drive is cheap and easily replaced (esp. if it's still under warranty). Your data on the other hand is not. I'd rather have a dead drive with the data backed up than a dead drive with no backup. Drives are so cheap nowadays it's a non-issue. I'm actually hoping my 2-year old drive dies soon because it will give me an excuse to go out and buy a *bigger* one for the same price. But I feel comfortable with that because I keep backups. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-19 18:18 ` Albert Hopkins @ 2009-11-20 16:03 ` Grant 2009-11-20 17:31 ` Albert Hopkins 0 siblings, 1 reply; 22+ messages in thread From: Grant @ 2009-11-20 16:03 UTC (permalink / raw To: gentoo-user >> > 4. Should I be comfortable running the entire sync operation every >> > night, or am I jeopardizing the longevity of my HDs? >> > >> This is a joke. > > I should apologize and explain this better. > > If you bought a fancy expensive hard drive then it's probably designed > for extremely heavy use and comes with nice coverage by the guy that > sold it to you. But since you ask this question I'll assume that it's > not the case. > > Ok so you bought a relatively cheap heard drive. But the question seems > irrelevant. Because these are cheap, mass-produced hard drives you > could do absolutely nothing with them and they could still die tomorrow. > That's the price you pay for cheaper drives. Cheap hard drives are like > life. Life is cheap. You could take all kinds of precautions and still > die tomorrow. But your drive is cheap and easily replaced (esp. if it's > still under warranty). Your data on the other hand is not. I'd rather > have a dead drive with the data backed up than a dead drive with no > backup. Drives are so cheap nowadays it's a non-issue. I'm actually > hoping my 2-year old drive dies soon because it will give me an excuse > to go out and buy a *bigger* one for the same price. But I feel > comfortable with that because I keep backups. Thank you for the clarification. Which are the "fancy expensive" hard drives? - Grant ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] rsync backup system 2009-11-20 16:03 ` Grant @ 2009-11-20 17:31 ` Albert Hopkins 0 siblings, 0 replies; 22+ messages in thread From: Albert Hopkins @ 2009-11-20 17:31 UTC (permalink / raw To: gentoo-user On Fri, 2009-11-20 at 08:03 -0800, Grant wrote: > Thank you for the clarification. Which are the "fancy expensive" hard > drives? The SAS drives that run at 15k RPM and cost $2-3 USD per GB. As opposed to your "run of the mill" 7200RPM SATA drive that costs pennies per GB. ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2010-03-17 15:34 UTC | newest] Thread overview: 22+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-11-19 16:44 [gentoo-user] rsync backup system Grant 2009-11-19 17:25 ` Neil Bothwick 2009-11-19 18:19 ` Grant 2009-11-20 17:05 ` Ward Poelmans 2009-11-20 16:05 ` Grant 2009-11-20 22:08 ` Neil Bothwick 2010-02-24 19:02 ` Grant 2010-02-24 20:51 ` Neil Bothwick 2010-02-25 15:15 ` Ward Poelmans 2010-02-25 15:41 ` Alan McKinnon 2010-02-25 16:11 ` Ward Poelmans 2010-02-25 17:50 ` daid kahl 2010-02-26 13:23 ` Ward Poelmans 2010-02-26 17:33 ` daid kahl 2010-02-25 17:52 ` Grant 2010-03-17 15:33 ` Ward Poelmans 2010-02-25 17:45 ` Neil Bothwick 2010-02-25 16:06 ` Neil Bothwick 2009-11-19 17:58 ` Albert Hopkins 2009-11-19 18:18 ` Albert Hopkins 2009-11-20 16:03 ` Grant 2009-11-20 17:31 ` Albert Hopkins
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox