ZenHAX

Free Game Research Forum - Official QuickBMS support - Twitter @zenhax
It is currently Fri Jan 20, 2017 4:10 pm

All times are UTC




Post new topic  Reply to topic  [ 13 posts ] 
Author Message
PostPosted: Sun Mar 15, 2015 4:44 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5374
http://aluigi.org/mytoolz.htm#offzip

Offzip is probably very known now, anyway it's a decompressor and scanner of zlib and deflate data... the classical algorithm used in the ZIP files and in the majority of games.

I have just improved the tool in the current 0.3.6 version, the following is the changelog:
Quote:
added the -c option that allows to guess and dump the chunked files, option -D to specify dictionary, -d to visualize the hexdump of the data before and after the compressed streams, statistics information, offset where the compressed streams ends, amount of bytes between the current compressed stream and the previous one, zlib header and crc information, updated extensions guesser (strnicmp fix for Linux)

From the point of view of who makes the scripts there are the ending offset and the amount of space between the streams that are very useful.

One of the most interesting features is just the -c option to guess the chunked files, so those files that are splitted in many compressed and non-compressed pieces.
Imagine to have a file splitted in the following way:
Code:
Offset      Compressed    Uncompressed
0x00000000: 0x00001000 -> 0x00004000
0x00001000: 0x00001000 -> 0x00004000
0x00002000: #no compressed data here
0x00003000: 0x00001000 -> 0x00004000
0x00004000: 0x00000080 -> 0x00000100
In this situation will be enough to use "offzip -a -c 0x4000" for being able to dump the original file.
Basically the tool thinks that the last chunk is the one smaller than the uncompressed chunk size (0x4000 in the example) and it will dump the non-compressed data if the space between the previous and next compressed stream is equal to the uncompressed chunk size.


Top
   
PostPosted: Fri Oct 23, 2015 6:06 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5374
I have just released Offzip 0.4.
The main bigger feature is the -r option to reimport the extracted files.
It works just like in quickbms so no need of presentations :D

This is the changelog:
Quote:
-r option for reimporting the extracted files like in QuickBMS (it uses the Zopfli library), the previous -r option has been renamed -R, some changes to the runtime help, it's no longer needed to specify the output folder and offset, option to automatically overwrite the output files, -1 now uses the output filename if specified, added a Makefile for Linux, renamed from Offset file unzipper to Offzip


Top
   
PostPosted: Mon Oct 26, 2015 11:16 pm 

Joined: Thu Oct 02, 2014 4:58 pm
Posts: 98
Testing the tool, i had some issues. :D


Top
   
PostPosted: Sun Nov 15, 2015 9:06 am 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 184
Can you add dump of compressed data? I need to dump all zlib blocks without decompression.

upd.

I'm blind, it's already realized with -A option :mrgreen:


Top
   
PostPosted: Sun Nov 15, 2015 10:16 am 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 184
Names for compressed data dumped from big files are invalid. They are only 8 characters offsets, but should be 16 characters.


Top
   
PostPosted: Sun Nov 15, 2015 12:02 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5374
Why? Offset 0x11223344 is 8 chars.
Offsets bigger than 4gb should be generated correctly if I remember correctly.


Top
   
PostPosted: Sun Nov 15, 2015 12:08 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 184
Nope, they are not correct. I have last zlib data on offset 0x1a9a5e004, but file was named a9a5e004.dat

And it doesn't work propperly for scanning too

Image


Top
   
PostPosted: Sun Nov 15, 2015 3:09 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5374
I see.
Basically the tool is ready for 64bit variables and file operations but some points are hardcoded to 32bit (name creation) or have a bug like readbase that uses "int" instead of i64.
Added in my TODO list.
Why are you using offzip on a so huge file? What game is that?


Top
   
PostPosted: Sun Nov 15, 2015 3:13 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 184
It's Darksiders II Deathinitive Edition. Will wait for update, thanks.


Top
   
PostPosted: Sun Nov 15, 2015 3:24 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5374
don't exist extractors/scripts for it?


Top
   
PostPosted: Sun Nov 15, 2015 3:37 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 184
Nope. And i don't want to extract it, i need compressed blocks and their offsets to reimport them back.


Top
   
PostPosted: Mon Nov 16, 2015 3:15 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 184
I've found another bug. Darksiders II Deathinitive Edition, file maps.upak (2.87gb). Offzip dumped compressed (-A option) file named 00001004.bod with size 4271842 bytes. I've copied block from same offset and with same size from original file and files are different.

On left side file that offzip dumped, on right original block.
http://puu.sh/lnEzY/86cd042225.png

Here is a sample. Even if you dump this file now with -A it have same size but different CRC.

http://www29.zippyshare.com/v/5tfzYga0/file.html


Top
   
PostPosted: Mon Nov 16, 2015 6:24 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5374
Ok, I will check it.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 13 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited