ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sun Feb 28, 2021 6:32 pm

All times are UTC




Post new topic  Reply to topic  [ 11 posts ] 
Author Message
PostPosted: Sun Feb 14, 2021 11:42 am 
User avatar

Joined: Sun Feb 14, 2021 11:36 am
Posts: 4
Hello,

We are working on the French translation of the game Tokyo Xanadu eX+ and we encounter a problem with quickbms (we tried with version 0.10.1 and version 0.11)

The game is composed of pkg files which are compressed with the NISLZS algorithm
The extraction goes smoothly and we get the files that are contained in the pkg files and they are perfectly readable

However, there is a problem with the import
Quickbms does not seem to compress the files when they are reinjected into the pkg file: the size of the "compressed" files is greater than the size of the uncompressed files

Example:
- open input file E:\xanadu_working\I_TITLEMENU.pkg
- open script E:\xanadu_working\tokyo_xanadu_ex+.bms
- set output folder E:\xanadu_working\files

offset filesize filename
--------------------------------------

Error: file "asset_D3D11.xml"
the reimport option acts as a reinjector, thereforey you cannot insert a
file if it's larger than the original for not overwriting the rest of
the archive which cannot be loaded correctly:

new size: 192 (188 uncompressed)
old size: 173 (188 uncompressed)

Error: file "titlemenu.dds.phyre"
the reimport option acts as a reinjector, thereforey you cannot insert a
file if it's larger than the original for not overwriting the rest of
the archive which cannot be loaded correctly:

new size: 4710923 (4196830 uncompressed)
old size: 548552 (4196830 uncompressed)

Is it a bug or is there something we didn't understand?
You will find attached the files used for this example

Thank you in advance for your help

Attachment:
tokyo_xanadu_ex-files.zip [680.16 KiB]
Downloaded 19 times


Top
   
PostPosted: Sun Feb 14, 2021 9:24 pm 
User avatar

Joined: Sun Feb 14, 2021 9:21 pm
Posts: 1
I work with Schneitizel on translation.

We forgot to specify that for the example above, the files were not modified between extraction and the insertion attempt.


Top
   
PostPosted: Sun Feb 14, 2021 11:37 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11999
It's normal when using reimport.bat (basic reinjector).

Use reimport2.bat and it will work ;)


Top
   
PostPosted: Mon Feb 15, 2021 6:04 am 
User avatar

Joined: Sun Feb 14, 2021 11:36 am
Posts: 4
aluigi wrote:
It's normal when using reimport.bat (basic reinjector).

Use reimport2.bat and it will work ;)


Hello, thanks for your reply!
We tried, and the result was... Quite predictable :p
Image
(When the game should load the modified file ; The .xml is next to a phyre, but if the .xml is not readable, the phyre is unlikely to be readable :/)
And for the .dds.phyre
Image

The .pkg goes from ~400Kb to ~5000Kb with the reimport2 :/


Top
   
PostPosted: Mon Feb 15, 2021 12:50 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11999
Ok, the problem is the double location of the compressed and uncompressed size values.
In reimport mode we can only change the location where the values used in the Clog command have been taken (in this case the entry).
The following script takes the SIZE/ZSIZE values from the referenced file itself, I don't know if the game will work but you can try using it in reimport mode:
Code:
comtype NISLZS

get TSTAMP long
get FILES long
for i = 0 < FILES
    getdstring NAME 0x40
    get SIZE long
    get ZSIZE long
    get OFFSET long
    get ZIP long

    if ZIP == 0
        log NAME OFFSET SIZE
    else
        math OFFSET + 8 # skip the repeated SIZE/ZSIZE
        get SIZE long  # fix
        get ZSIZE long # fix
        clog NAME OFFSET ZSIZE SIZE
    endif
next i


The NISLZS compression uses a fake recompressor that takes double the size since I don't have the original compression code.
In reimport mode the files are appended at the end of the archive if the reimported file is bigger than the original once "stored" back (so compressed or as-is depending by the format).

If even with the new script the game refuses to run, you can use a special feature of the new quickbms beta.
Use the original script (not the new one) and create a copy of reimport2.bat adding the -e option in it, then use it again for reimporting.
The -e option tries to disable the compression used in archives so in this case it will set the ZIP field to zero and store the file as-is.

quickbms beta is available here:
https://aluigi.org/beta/quickbms_exe.zip

This is the content of the new reimport2.bat:
Code:
quickbms.exe -G -w -r -r -e

Please let us know what method works.


Top
   
PostPosted: Fri Feb 19, 2021 12:33 pm 
User avatar

Joined: Sun Feb 14, 2021 11:36 am
Posts: 4
Hi aluigi! Thanks for your reply, and sorry for the late answer, we tried a lot of things

The "fix" script for reimport2 does not give interesting results: the generated pkg is 1.8 GB

Test with the beta version of quickbms :

The generated I_TITLEMENU.pkg file is 4 635 Kb, the size is correct (base size + addition of the 2 uncompressed files)
On the other hand, the offsets are wrong because they start 8 bytes before the start of the data
So I manually corrected the offsets to make it right, but it doesn't work ingame

The game doesn't crash but is not able to display the image

Image
Here is the normal result, in case
Image

When we set the ZIP at 02 instead of 00
Image

In attachment, you will find the pkg generated with the beta version, as well as the pkg whose offsets I modified


Attachments:
I_TITLEMENU pkgs.rar [496.96 KiB]
Downloaded 8 times
Top
   
PostPosted: Sat Feb 20, 2021 10:58 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11999
Ok so the 8 bytes difference is definitely a bug in quickbms... I will fix it :)

The other error is a bit strange because the first hash listed there is the beginning of the xml file ("<?xm").
I suppose also this problem is related to a bug in the reimport feature but I don't understand where it gets the second hash since the archive doesn't have any.
I will check.


Top
   
PostPosted: Fri Feb 26, 2021 9:22 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11999
The fault was the new script because it was totally wrong... sorry for that.
But there is definitely something else wrong probably in quickbms that needs to be fixed, I'm checking it right now.


Top
   
PostPosted: Fri Feb 26, 2021 9:27 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11999
Update: ok I think I understand what's wrong but I don't think there is any solution.

The entry table has OFFSET, SIZE and ZSIZE.
OFFSET points to a container format where is located another copy of SIZE and ZSIZE.

The first problem is that quickbms can't modify both the copies of SIZE and ZSIZE, just the last one.

Now, your reimported file will be bigger than the original because nislzs is a fake recompression, so OFFSET will point to the new location at the end of the file.
BUT SIZE and ZSIZE still point to the old location!

From my side there is nothing I can do unfortunately.


Top
   
PostPosted: Sat Feb 27, 2021 8:20 pm 
User avatar

Joined: Sun Feb 14, 2021 11:36 am
Posts: 4
Hi Aluigi, thanks again for you work and sorry again for the late answer xD
We found a solution, just drag the folder containing the .phyre and .xml into PKGToolCmd.exe, provided with SenPkgTool

However, extraction with QuickBMS works perfectly :)
https://github.com/Sewer56/Sen-no-Kiseki-PKG-Sharp

Thanks again for your time, which put us on the track, and hopefully this topic will at least bring a little to QuickBMS :)

Image


Top
   
PostPosted: Sun Feb 28, 2021 11:27 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11999
That's a great news! :D


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 11 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited