[gentoo-user] problem reading zip files

[gentoo-user] problem reading zip files

[Doug ONeal]

I have had a problem dumped on me that I cannot resolve.  A user
archived some very large
datasets using Info-zip (app-arch/zip-2.32-r1) and the original data has
been deleted.  The zip
executable is a 64-bit binary and the user assumed that large archives
were possible.  A typical
archive is on the order of 70GB and the largest is over 200GB
(compressed sizes).  /usr/bin/zip did
not complain about creating the archive and /usr/bin/unzip -l lists the
contents correctly so the user
did not think there were any problems until it came time to extract a
file.  Extracting any file that
is past the 4GB mark in the archive results in a 'bad zipfile offset'
error.  This is consistent with the
FAQ on Info-zip's web site.

I am guessing that all of the data is actually present in the zip
archives but I cannot get to it.  Trying
version 3.0 of info-zip or any of the other linux & windows unzip
programs has not been successful
in extracting any of the later files in the archives.   I'm at a loss;
does anybody have suggestions on
how to retrieve this data?  Thanks.

Doug
--
Douglas O'Neal
Manager, Bioinformatics Core Center
Delaware Biotechnology Institute

Re: [gentoo-user] problem reading zip files

[Robert Bridge]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Doug ONeal wrote:
> I have had a problem dumped on me that I cannot resolve.  A user
> archived some very large
> datasets using Info-zip (app-arch/zip-2.32-r1) and the original data has
> been deleted.  The zip
> executable is a 64-bit binary and the user assumed that large archives
> were possible.  A typical
> archive is on the order of 70GB and the largest is over 200GB
> (compressed sizes).  /usr/bin/zip did
> not complain about creating the archive and /usr/bin/unzip -l lists the
> contents correctly so the user
> did not think there were any problems until it came time to extract a
> file.  Extracting any file that
> is past the 4GB mark in the archive results in a 'bad zipfile offset'
> error.  This is consistent with the
> FAQ on Info-zip's web site.
> 
> I am guessing that all of the data is actually present in the zip
> archives but I cannot get to it.  Trying
> version 3.0 of info-zip or any of the other linux & windows unzip
> programs has not been successful
> in extracting any of the later files in the archives.   I'm at a loss;
> does anybody have suggestions on
> how to retrieve this data?  Thanks.

The second thought that springs to would be be having a look at the
source code for unzip and seeing if you can tweak the "zipfile offset"
to allow for larger values than normal.

Assuming the data is there and correctly compressed, it should just be a
matter of getting the program to read it.

RobbieAB
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpDgAYACgkQZr0UhZgPVmya5QCePiJfsPo60LXuSh2d4RUj5CHv
BZcAn2Y9UUm42oAfd2IXj8RiqQcrXDc0
=Yeml
-----END PGP SIGNATURE-----

Re: [gentoo-user] problem reading zip files

[Arttu V.]

On 6/25/09, Doug ONeal <oneal@dbi.udel.edu> wrote:
> I am guessing that all of the data is actually present in the zip
> archives but I cannot get to it.  Trying
> version 3.0 of info-zip or any of the other linux & windows unzip
> programs has not been successful
> in extracting any of the later files in the archives.   I'm at a loss;
> does anybody have suggestions on
> how to retrieve this data?  Thanks.

Does "any of the other unzip programs" here contain java's jar as well?

-- 
Arttu V.

[gentoo-user] Re: problem reading zip files

[Doug O'Neal]

On 06/25/2009 09:51 AM, Arttu V. wrote:
> On 6/25/09, Doug ONeal <oneal@dbi.udel.edu> wrote:
>> I am guessing that all of the data is actually present in the zip
>> archives but I cannot get to it.  Trying
>> version 3.0 of info-zip or any of the other linux & windows unzip
>> programs has not been successful
>> in extracting any of the later files in the archives.   I'm at a loss;
>> does anybody have suggestions on
>> how to retrieve this data?  Thanks.
> 
> Does "any of the other unzip programs" here contain java's jar as well?
> 

No, I didn't think of trying jar.  The process is somewhat slow but I
have extracted files from a 58GB archive well past the 4GB limit
and that unzip wouldn't touch.  jar has taken 25 minutes to extract
16GB of data so it may be a while before the process completes but
it looks like this is a solution.  Thanks.

Re: [gentoo-user] Re: problem reading zip files

[Paul Hartman]

On Thu, Jun 25, 2009 at 9:45 AM, Doug O'Neal<oneal@dbi.udel.edu> wrote:
> On 06/25/2009 09:51 AM, Arttu V. wrote:
>> On 6/25/09, Doug ONeal <oneal@dbi.udel.edu> wrote:
>>> I am guessing that all of the data is actually present in the zip
>>> archives but I cannot get to it.  Trying
>>> version 3.0 of info-zip or any of the other linux & windows unzip
>>> programs has not been successful
>>> in extracting any of the later files in the archives.   I'm at a loss;
>>> does anybody have suggestions on
>>> how to retrieve this data?  Thanks.
>>
>> Does "any of the other unzip programs" here contain java's jar as well?
>>
>
> No, I didn't think of trying jar.  The process is somewhat slow but I
> have extracted files from a 58GB archive well past the 4GB limit
> and that unzip wouldn't touch.  jar has taken 25 minutes to extract
> 16GB of data so it may be a while before the process completes but
> it looks like this is a solution.  Thanks.

It seems you've already solved your problem, but I'm curious to know
if you tried 7-Zip. It is usually able to deal with just about
anything.

[gentoo-user] Re: problem reading zip files

[Doug O'Neal]

On 06/25/2009 11:31 AM, Paul Hartman wrote:
> On Thu, Jun 25, 2009 at 9:45 AM, Doug O'Neal<oneal@dbi.udel.edu> wrote:
>> On 06/25/2009 09:51 AM, Arttu V. wrote:
>>> On 6/25/09, Doug ONeal <oneal@dbi.udel.edu> wrote:
>>>> I am guessing that all of the data is actually present in the zip
>>>> archives but I cannot get to it.  Trying
>>>> version 3.0 of info-zip or any of the other linux & windows unzip
>>>> programs has not been successful
>>>> in extracting any of the later files in the archives.   I'm at a loss;
>>>> does anybody have suggestions on
>>>> how to retrieve this data?  Thanks.
>>> Does "any of the other unzip programs" here contain java's jar as well?
>>>
>> No, I didn't think of trying jar.  The process is somewhat slow but I
>> have extracted files from a 58GB archive well past the 4GB limit
>> and that unzip wouldn't touch.  jar has taken 25 minutes to extract
>> 16GB of data so it may be a while before the process completes but
>> it looks like this is a solution.  Thanks.
> 
> It seems you've already solved your problem, but I'm curious to know
> if you tried 7-Zip. It is usually able to deal with just about
> anything.

7-Zip did not work.  I also tried the fuse-zip filesystem and it didn't even
recognize the files as zips.

Re: [gentoo-user] Re: problem reading zip files

[Paul Hartman]

On Thu, Jun 25, 2009 at 10:47 AM, Doug O'Neal<oneal@dbi.udel.edu> wrote:
> On 06/25/2009 11:31 AM, Paul Hartman wrote:
>> On Thu, Jun 25, 2009 at 9:45 AM, Doug O'Neal<oneal@dbi.udel.edu> wrote:
>>> On 06/25/2009 09:51 AM, Arttu V. wrote:
>>>> On 6/25/09, Doug ONeal <oneal@dbi.udel.edu> wrote:
>>>>> I am guessing that all of the data is actually present in the zip
>>>>> archives but I cannot get to it.  Trying
>>>>> version 3.0 of info-zip or any of the other linux & windows unzip
>>>>> programs has not been successful
>>>>> in extracting any of the later files in the archives.   I'm at a loss;
>>>>> does anybody have suggestions on
>>>>> how to retrieve this data?  Thanks.
>>>> Does "any of the other unzip programs" here contain java's jar as well?
>>>>
>>> No, I didn't think of trying jar.  The process is somewhat slow but I
>>> have extracted files from a 58GB archive well past the 4GB limit
>>> and that unzip wouldn't touch.  jar has taken 25 minutes to extract
>>> 16GB of data so it may be a while before the process completes but
>>> it looks like this is a solution.  Thanks.
>>
>> It seems you've already solved your problem, but I'm curious to know
>> if you tried 7-Zip. It is usually able to deal with just about
>> anything.
>
> 7-Zip did not work.  I also tried the fuse-zip filesystem and it didn't even
> recognize the files as zips.

Good to know. I'll have to keep jar in mind for future reference. :)
Thanks to Arttu for the tip.