ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Thu Oct 22, 2020 8:27 pm

All times are UTC




Post new topic  Reply to topic  [ 2 posts ] 
Author Message
PostPosted: Sat Sep 26, 2020 7:54 pm 

Joined: Wed Sep 09, 2020 10:26 am
Posts: 2
I came here earlier and posted my progress with this tool in the totally wrong board, and in there I said I was thinking of writing a compression algorithm... well, here it is.

To recap: I reversed the decompression function for Okage: Shadow King, and made a tool in Python for extracting the assets in the game's .XPF archives.
Now there's an options to pack the assets back into .XPF archives, because I wrote a compression function based on the decompressor.
It recompresses decompressed files just as well (tends to actually produce identical compression streams), and even better (even smaller files), than the original tool the developers used.

Attached is the tool written in Python. When fed an .XPF file, it will extract and decompress all contained files. When fed a folder, it will compress all files and pack them into an .XPF file.
Also included are C++ implementations of the same decompression and compression functions standalone, because I simply don't know to make an equivalent tool in this language - but I made these out of necessity because speed is an issue with Python; it may very well take hours to compress 1 file, while CPP takes mere seconds to do the same. See the recompression table under EDIT 2.

EDIT: Fixed a blunder that would confound anyone reading the compression algorithm. Also attempted to speed up the Python version a bit, and added a readme to help you use it.
EDIT 2: Added a compression levels option, which includes medium compression in a reasonable timeframe, and an option to completely match the compression in the original game files.
I selected this big file below to demonstrate the different levels. It's probably in the extreme end of time differences, but that also serves to clearly show what the levels are capable of.
Code:
Recompression table for DATA\BTLDATA\ALPHA\FIELD\BEILY41.XPF using OSK Tool.py
Level   Elapsed   Compressed size
0       0.02s     346,764 (No compression.)
1       9.6s      111,640 (Low compression; short writes, short seeks.)
2       9.6s      111,585 (Shallow compression; long writes, short seeks.)
3       128.14s   94,211  (Identical game compression; short writes, long seeks.)
4       118.21s   93,716  (Maximum compression; long writes and long seeks.)


Attachments:
File comment: Includes: OSK Tool.py, OSK Decompress.cpp, OSK Compress.cpp, readme.txt
OSK Tool and code.zip [7.54 KiB]
Downloaded 13 times
Top
   
PostPosted: Sat Oct 10, 2020 11:59 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11688
Well done and thanks for the code.
I will support this compression in quickbms 0.11, I just added it to the beta today and I will update the public beta next week.
In quickbms I will not handle the 32bit field at the beginning because I prefer to make the algorithm usable in the comtype_scan2 scanning.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 2 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited