From: Mark Knecht <markknecht@gmail.com>
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Use split to break up a 10GB file binary?
Date: Mon, 20 Jun 2011 11:20:03 -0700 [thread overview]
Message-ID: <BANLkTimU1MxR+cF+SkjxOy+zSo=hFoqiRw@mail.gmail.com> (raw)
In-Reply-To: <yu9tybkwg0l.fsf@nyu.edu>
On Mon, Jun 20, 2011 at 10:48 AM, Allan Gottlieb <gottlieb@nyu.edu> wrote:
> On Mon, Jun 20 2011, Paul Hartman wrote:
>
>> On Mon, Jun 20, 2011 at 10:25 AM, Mark Knecht <markknecht@gmail.com> wrote:
>>> Hi,
>>> Is split an appropriate program to use to break a single 10GB file
>>> into 100 100MB files to transfer over the net using rsync, and then
>>> use cat to reassemble?
>>
>> I think it should work just fine. I've split huge files into huge
>> chunks and never had any issues.
>>
>>> Is there some better way to do this?
>>
>> I wonder if splitting is even necessary; rsync will analyze the file
>> and only transmit the differences, right?. So I'd think that even if
>> the transfer fails, a retry would pick up where it left off (assuming
>> rsync keeps the failed copy).
>
> I believe that is the --partial option.
>
> allan
Yes, that looks like what I want.
Is there an option to have rsync keep trying if the other end goes
down for a while or would I need to put the rsync command into a cron
job so that it restarts every hour until it's completed the transfers?
I don't see one scanning through the many, many options to rsync.
Thanks,
Mark
next prev parent reply other threads:[~2011-06-20 19:04 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-06-20 15:25 [gentoo-user] Use split to break up a 10GB file binary? Mark Knecht
2011-06-20 16:21 ` Paul Hartman
2011-06-20 16:30 ` Mark Knecht
2011-06-20 17:51 ` Michael Hampicke
2011-06-20 17:48 ` Allan Gottlieb
2011-06-20 18:20 ` Mark Knecht [this message]
2011-06-21 7:08 ` [gentoo-user] " Francesco Talamona
2011-06-20 19:17 ` [gentoo-user] " Nils Larsson
2011-06-20 19:29 ` Mark Knecht
2011-06-20 20:51 ` Walter Dnes
2011-06-20 21:06 ` Mark Knecht
2011-06-20 21:58 ` Mick
2011-06-20 21:46 ` Indi
2011-06-21 0:28 ` Mark Knecht
2011-06-21 10:41 ` Indi
2011-06-21 15:11 ` Peter Humphrey
2011-06-21 15:20 ` Dale
2011-06-21 15:36 ` Mark Knecht
2011-06-21 17:00 ` Peter Humphrey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='BANLkTimU1MxR+cF+SkjxOy+zSo=hFoqiRw@mail.gmail.com' \
--to=markknecht@gmail.com \
--cc=gentoo-user@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox