ZenHAX
http://zenhax.com/

Offset file unzipper (offzip)
http://zenhax.com/viewtopic.php?f=17&t=728
Page 1 of 1

Author:  aluigi [ Sun Mar 15, 2015 4:44 pm ]
Post subject:  Offset file unzipper (offzip)

http://aluigi.org/mytoolz.htm#offzip

Offzip is probably very known now, anyway it's a decompressor and scanner of zlib and deflate data... the classical algorithm used in the ZIP files and in the majority of games.

I have just improved the tool in the current 0.3.6 version, the following is the changelog:
Quote:
added the -c option that allows to guess and dump the chunked files, option -D to specify dictionary, -d to visualize the hexdump of the data before and after the compressed streams, statistics information, offset where the compressed streams ends, amount of bytes between the current compressed stream and the previous one, zlib header and crc information, updated extensions guesser (strnicmp fix for Linux)

From the point of view of who makes the scripts there are the ending offset and the amount of space between the streams that are very useful.

One of the most interesting features is just the -c option to guess the chunked files, so those files that are splitted in many compressed and non-compressed pieces.
Imagine to have a file splitted in the following way:
Code:
Offset      Compressed    Uncompressed
0x00000000: 0x00001000 -> 0x00004000
0x00001000: 0x00001000 -> 0x00004000
0x00002000: #no compressed data here
0x00003000: 0x00001000 -> 0x00004000
0x00004000: 0x00000080 -> 0x00000100
In this situation will be enough to use "offzip -a -c 0x4000" for being able to dump the original file.
Basically the tool thinks that the last chunk is the one smaller than the uncompressed chunk size (0x4000 in the example) and it will dump the non-compressed data if the space between the previous and next compressed stream is equal to the uncompressed chunk size.

Author:  aluigi [ Fri Oct 23, 2015 6:06 pm ]
Post subject:  Re: Offset file unzipper (offzip)

I have just released Offzip 0.4.
The main bigger feature is the -r option to reimport the extracted files.
It works just like in quickbms so no need of presentations :D

This is the changelog:
Quote:
-r option for reimporting the extracted files like in QuickBMS (it uses the Zopfli library), the previous -r option has been renamed -R, some changes to the runtime help, it's no longer needed to specify the output folder and offset, option to automatically overwrite the output files, -1 now uses the output filename if specified, added a Makefile for Linux, renamed from Offset file unzipper to Offzip

Author:  Savage [ Mon Oct 26, 2015 11:16 pm ]
Post subject:  Re: Offset file unzipper (offzip)

Testing the tool, i had some issues. :D

Author:  spider91 [ Sun Nov 15, 2015 9:06 am ]
Post subject:  Re: Offset file unzipper (offzip)

Can you add dump of compressed data? I need to dump all zlib blocks without decompression.

upd.

I'm blind, it's already realized with -A option :mrgreen:

Author:  spider91 [ Sun Nov 15, 2015 10:16 am ]
Post subject:  Re: Offset file unzipper (offzip)

Names for compressed data dumped from big files are invalid. They are only 8 characters offsets, but should be 16 characters.

Author:  aluigi [ Sun Nov 15, 2015 12:02 pm ]
Post subject:  Re: Offset file unzipper (offzip)

Why? Offset 0x11223344 is 8 chars.
Offsets bigger than 4gb should be generated correctly if I remember correctly.

Author:  spider91 [ Sun Nov 15, 2015 12:08 pm ]
Post subject:  Re: Offset file unzipper (offzip)

Nope, they are not correct. I have last zlib data on offset 0x1a9a5e004, but file was named a9a5e004.dat

And it doesn't work propperly for scanning too

Image

Author:  aluigi [ Sun Nov 15, 2015 3:09 pm ]
Post subject:  Re: Offset file unzipper (offzip)

I see.
Basically the tool is ready for 64bit variables and file operations but some points are hardcoded to 32bit (name creation) or have a bug like readbase that uses "int" instead of i64.
Added in my TODO list.
Why are you using offzip on a so huge file? What game is that?

Author:  spider91 [ Sun Nov 15, 2015 3:13 pm ]
Post subject:  Re: Offset file unzipper (offzip)

It's Darksiders II Deathinitive Edition. Will wait for update, thanks.

Author:  aluigi [ Sun Nov 15, 2015 3:24 pm ]
Post subject:  Re: Offset file unzipper (offzip)

don't exist extractors/scripts for it?

Author:  spider91 [ Sun Nov 15, 2015 3:37 pm ]
Post subject:  Re: Offset file unzipper (offzip)

Nope. And i don't want to extract it, i need compressed blocks and their offsets to reimport them back.

Author:  spider91 [ Mon Nov 16, 2015 3:15 pm ]
Post subject:  Re: Offset file unzipper (offzip)

I've found another bug. Darksiders II Deathinitive Edition, file maps.upak (2.87gb). Offzip dumped compressed (-A option) file named 00001004.bod with size 4271842 bytes. I've copied block from same offset and with same size from original file and files are different.

On left side file that offzip dumped, on right original block.
http://puu.sh/lnEzY/86cd042225.png

Here is a sample. Even if you dump this file now with -A it have same size but different CRC.

http://www29.zippyshare.com/v/5tfzYga0/file.html

Author:  aluigi [ Mon Nov 16, 2015 6:24 pm ]
Post subject:  Re: Offset file unzipper (offzip)

Ok, I will check it.

Author:  aluigi [ Mon Mar 12, 2018 6:22 pm ]
Post subject:  Re: Offset file unzipper (offzip)

Sorry for necroposting but I don't think I can replicate the issue, just made a simple test (garbage+zlib+garbage) since I no longer have the sample you provided.

Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/