Re: [gentoo-user] Use split to break up a 10GB file binary?

[Mark Knecht <markknecht@gmail.com>]

On Mon, Jun 20, 2011 at 1:51 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Mon, Jun 20, 2011 at 08:25:57AM -0700, Mark Knecht wrote
>> Hi,
>>    Is split an appropriate program to use to break a single 10GB file
>> into 100 100MB files to transfer over the net using rsync, and then
>> use cat to reassemble?
>>
>>    Is there some better way to do this?
>
>  That's what split was written for.  I can't think of anything better.
> BTW, what type of data is the 10 gig file?  If it's text, then consider
> using zip or bzip2 on each of the fragments before transferring.  If
> it's an already compressed binary format, then don't waste time
> attempting further compression.
>
> --
> Walter Dnes <waltdnes@waltdnes.org>

Hi Walter.

It's a Virtualbox VM exported from virtualbox-4.0.8. It consumes about
40GB on disk, compressed by Virtualbox to about 10GB by their 'Export
Appliance' feature.

I actually did try compressing the exported 10GB file with gzip &
bzip2 before splitting it. None of those provided any compression. I
didn't try the spilt outputs as I figured they are just binary chunks
and wouldn't compress either.

Thanks for the info.

Cheers,
Mark