From mboxrd@z Thu Jan 1 00:00:00 1970
Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org)
by finch.gentoo.org with esmtp (Exim 4.60)
(envelope-from <gentoo-user+bounces-124486-garchives=archives.gentoo.org@lists.gentoo.org>)
id 1QYpPy-0004TW-5X
for garchives@archives.gentoo.org; Tue, 21 Jun 2011 01:06:14 +0000
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id C22A01C04C
for <garchives@archives.gentoo.org>; Tue, 21 Jun 2011 01:06:13 +0000 (UTC)
Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.216.181])
by pigeon.gentoo.org (Postfix) with ESMTP id C75281C040
for <gentoo-user@lists.gentoo.org>; Tue, 21 Jun 2011 00:28:05 +0000 (UTC)
Received: by qyk9 with SMTP id 9so1393246qyk.19
for <gentoo-user@lists.gentoo.org>; Mon, 20 Jun 2011 17:28:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;
h=domainkey-signature:mime-version:in-reply-to:references:date
:message-id:subject:from:to:content-type:content-transfer-encoding;
bh=M1sRg8AYC7O9twyPv4uIL8PnHr6jKk74o/q5kr5Nx4E=;
b=oOfp0X4YvyaYCoT73+Fj5L4bbjhi9U3ywbpZlqN25JlKnIutAj/qyf6KHpLkPG7JMk
eIs61krDjNkZBnykFH/DCnx2aJuMiPzpglwHCAvzKRnOOTR9OJ0ybKzYpZH7db4w/kEe
9C8Ix5FZqF8z5RUZj8MVsOYtFkM584+0OqtVQ=
DomainKey-Signature: a=rsa-sha1; c=nofws;
d=gmail.com; s=gamma;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:content-type:content-transfer-encoding;
b=tWXiYabK6/Bj74tUkgLcfebi0V7IxSUmtgEyjBXGDjN9dWAU7U9szzFk9Nc4Ich9r7
LQuxbwhr8KGvL1E1Smm03fsiYbMIrdezO2XaNZ7x71ynob8/pi5FPp2y4g0UhrzKBFcp
Kk6Hbo/OqjASUvpF0fIHjTegJfG+2l7YNbUu4=
Precedence: bulk
List-Post: <mailto:gentoo-user@lists.gentoo.org>
List-Help: <mailto:gentoo-user+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-user+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-user+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-user.gentoo.org>
X-BeenThere: gentoo-user@lists.gentoo.org
Reply-to: gentoo-user@lists.gentoo.org
MIME-Version: 1.0
Received: by 10.224.127.66 with SMTP id f2mr4013878qas.28.1308616085192; Mon,
20 Jun 2011 17:28:05 -0700 (PDT)
Received: by 10.224.61.14 with HTTP; Mon, 20 Jun 2011 17:28:05 -0700 (PDT)
In-Reply-To: <20110620214602.GA7448@gaurahari.merseine.nu>
References: <BANLkTinPvM8-YeNgQOPggL=JvZJvrOMuWw@mail.gmail.com>
<20110620214602.GA7448@gaurahari.merseine.nu>
Date: Mon, 20 Jun 2011 17:28:05 -0700
Message-ID: <BANLkTimCOZacwvQMNCYp68YiyM1U73A4Xw@mail.gmail.com>
Subject: Re: [gentoo-user] Use split to break up a 10GB file binary?
From: Mark Knecht <markknecht@gmail.com>
To: gentoo-user@lists.gentoo.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Archives-Salt:
X-Archives-Hash: 3646b6989ec22bd5efb722f6dbf1e870
On Mon, Jun 20, 2011 at 2:46 PM, Indi <thebeelzebubtrigger@gmail.com> wrote=
:
> On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote:
>> Hi,
>> =C2=A0 =C2=A0Is split an appropriate program to use to break a single 10=
GB file
>> into 100 100MB files to transfer over the net using rsync, and then
>> use cat to reassemble?
>>
>> =C2=A0 =C2=A0Is there some better way to do this?
>>
>
> Just using rsync by itself would probably be a great deal faster,
> unless you have some undisclosed reason for wanting to split it up.
Hi,
Nothing technical that's undisclosed. My original reason was not
knowing what rsync did in the case of errors I simply didn't want to
start over on such a big file. I figured there was little to lose by
stitching it back together are the other end and I could always figure
out exactly what file had failed.
That said I don't think there's much difference in the speed. In my
case (and I think others will have a similar case) my uploads speeds
are far lower than download. I get about 8MB/S download but only about
250KB/S upload. It's that low speed that's dominating everything else.
When I first tried transferring the big file the intermediate speeds
rsync was reporting were very similar.
Cheers,
Mark