From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org)
	by finch.gentoo.org with esmtp (Exim 4.60)
	(envelope-from <gentoo-user+bounces-124457-garchives=archives.gentoo.org@lists.gentoo.org>)
	id 1QYhtL-000797-UX
	for garchives@archives.gentoo.org; Mon, 20 Jun 2011 17:04:04 +0000
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
	by pigeon.gentoo.org (Postfix) with SMTP id DE6EA1C098
	for <garchives@archives.gentoo.org>; Mon, 20 Jun 2011 17:04:02 +0000 (UTC)
Received: from mail-vx0-f181.google.com (mail-vx0-f181.google.com [209.85.220.181])
	by pigeon.gentoo.org (Postfix) with ESMTP id 9B3EC1C044
	for <gentoo-user@lists.gentoo.org>; Mon, 20 Jun 2011 16:22:19 +0000 (UTC)
Received: by vxa40 with SMTP id 40so1382475vxa.40
        for <gentoo-user@lists.gentoo.org>; Mon, 20 Jun 2011 09:22:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:sender:in-reply-to:references:from
         :date:x-google-sender-auth:message-id:subject:to:content-type
         :content-transfer-encoding;
        bh=O//K3sQV+RCYvJSE1Cyny2dhiguWRJQ7oB0dVLanhyM=;
        b=nVteNHhNOs9OvLavCdnp+mx8Yo31581yjT9pqfzmqMnCSAjhP3XQZMo4DFoSaivGUk
         RsnbmZbGWlQo8ZNqXavaK5ndTVZX5Kb0kN/7Fkw4XrUPFyMeHDXnLGcXxqiiATbCJefg
         jJ5NMw9yr81tKScJh3/zWTHIlfDvQqBUlL7v0=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:from:date
         :x-google-sender-auth:message-id:subject:to:content-type
         :content-transfer-encoding;
        b=to36AiLdaYYooaed++9bpxs3eWcFBSvOM2oDD7nAcj2HHLPwzCIdsPAQOU0ym29zsX
         bRVCpeKhRiEnlbOremEH8A5SVSMp3EX+vMB4mjikVfbdGkuEneObnAa2r+boAM7jnBbb
         f/5sQxmeD0CAgAb4JoX9Hd1GJgpOQF2VDn78I=
Received: by 10.52.73.67 with SMTP id j3mr3583463vdv.75.1308586939123; Mon, 20
 Jun 2011 09:22:19 -0700 (PDT)
Precedence: bulk
List-Post: <mailto:gentoo-user@lists.gentoo.org>
List-Help: <mailto:gentoo-user+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-user+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-user+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-user.gentoo.org>
X-BeenThere: gentoo-user@lists.gentoo.org
Reply-to: gentoo-user@lists.gentoo.org
MIME-Version: 1.0
Sender: paul.hartman@gmail.com
Received: by 10.52.157.41 with HTTP; Mon, 20 Jun 2011 09:21:59 -0700 (PDT)
In-Reply-To: <BANLkTinPvM8-YeNgQOPggL=JvZJvrOMuWw@mail.gmail.com>
References: <BANLkTinPvM8-YeNgQOPggL=JvZJvrOMuWw@mail.gmail.com>
From: Paul Hartman <paul.hartman+gentoo@gmail.com>
Date: Mon, 20 Jun 2011 11:21:59 -0500
X-Google-Sender-Auth: PB9ZJG0umNvdDvijfQ650lMQBus
Message-ID: <BANLkTi=3Ai7xndu+sNVCxLPKD=z8qE5vJA@mail.gmail.com>
Subject: Re: [gentoo-user] Use split to break up a 10GB file binary?
To: gentoo-user@lists.gentoo.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Archives-Salt: 
X-Archives-Hash: 6d84ec3d0edc4ea07b9364094c7731f2

On Mon, Jun 20, 2011 at 10:25 AM, Mark Knecht <markknecht@gmail.com> wrote:
> Hi,
> =A0 Is split an appropriate program to use to break a single 10GB file
> into 100 100MB files to transfer over the net using rsync, and then
> use cat to reassemble?

I think it should work just fine. I've split huge files into huge
chunks and never had any issues.

> =A0 Is there some better way to do this?

I wonder if splitting is even necessary; rsync will analyze the file
and only transmit the differences, right?. So I'd think that even if
the transfer fails, a retry would pick up where it left off (assuming
rsync keeps the failed copy).

Also check out net-misc/unison. It seems to be designed for just this
sort of thing.