ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Thu Apr 15, 2021 7:42 am

All times are UTC




Post new topic  Reply to topic  [ 9 posts ] 
Author Message
PostPosted: Wed Mar 31, 2021 9:35 pm 

Joined: Thu Nov 26, 2020 2:13 pm
Posts: 18
Code:
# KINGDOM HEARTS HD 1.5 + 2.5 ReMIX (Microsoft Windows)
# KINGDOM HEARTS HD 2.8 Final Chapter Prologue (Microsoft Windows)

math is_hed = 0
math is_pkg = 0

get ext extension
if ext == "hed"
   math is_pkg = 1
   callfunction handle_hed_pkg 1
elif ext == "pkg"
   math is_hed = 1
   callfunction handle_hed_pkg 1
endif

startfunction handle_hed_pkg
   open FDDE "hed" is_hed
   open FDDE "pkg" is_pkg
   get hed_size asize is_hed
   xmath hed_files "hed_size / 0x20"
   for i = 0 < hed_files
      set name string ""
      getdstring hash 0x10 is_hed
      string hash_in_str b hash
      putvarchr hash_in_str 32 0
      get file_offset longlong is_hed
      get file_size1 long is_hed
      get file_size2 long is_hed
      set name string hash_in_str
      goto file_offset is_pkg
      get file_size3 long is_pkg
      get sub_entries long is_pkg
      get compressed_file_size1 long is_pkg
      get file_info3 long is_pkg
      if sub_entries == 0
         string name + "."
         math file_offset + 16
         if compressed_file_size1 <= 0xfffffff0
            log name file_offset compressed_file_size1 is_pkg
         else
            log name file_offset file_size3 is_pkg
         endif
      else
         string name1 p "%s/init." name
         xmath helper "16 + (sub_entries * 0x30)"
         math file_offset + helper
         if compressed_file_size1 <= 0xfffffff0
            log name1 file_offset compressed_file_size1 is_pkg
            math file_offset + compressed_file_size1
         else
            log name1 file_offset file_size3 is_pkg
            math file_offset + file_size3
         endif
         for j = 0 < sub_entries
            string name2 p "%s/%u." name j
            getdstring sbe_name 0x20 is_pkg
            get sbe1 long is_pkg
            get sbe2 long is_pkg
            get sbe3 long is_pkg
            get sbe4 long is_pkg
            if sbe4 <= 0xfffffff0
               log name2 file_offset sbe4 is_pkg
               math file_offset + sbe4
            else
               log name2 file_offset sbe3 is_pkg
               xmath sbe3_1 "sbe3 && 0x10"
               math file_offset + sbe3_1
            endif
         next j
      endif
   next i
endfunction
this script can only extract files as-is, as a consequence it cannot handle any decompression stuff whatsoever. this is why you get some garbled files.


Top
   
PostPosted: Thu Apr 01, 2021 4:39 pm 

Joined: Thu Apr 01, 2021 4:32 pm
Posts: 2
Hello tbmq008.
I have a project to translate the game into Portuguese-BR and I am looking for tips on how to extract the texts from the game.
Use the bms with the script above with the files "SettingMenu.hed" and "SettingMenu.pkg".
When extracting, I have the files ".sed", ".l2d" and ".dat".
Any ideas on how to extract texts from menus and dialogues?

Thanks.


Top
   
PostPosted: Thu Apr 01, 2021 5:24 pm 

Joined: Thu Nov 26, 2020 2:13 pm
Posts: 18
".sed" files are simply sound data.
as for the rest i have no idea, i went in and wrote the script in "blind mode".


Top
   
PostPosted: Sat Apr 03, 2021 11:19 am 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 7
As far as i can see all locale files starts with "@CTD", then some metadata and then UTF-16 encoded strings.
Replacing strings should be easy if someone figures out that metadata before strings :)

I've uploaded file where you can see results of "translating" this file to blah language :D (it's the settings menu in launcher),

Image


Attachments:
File comment: Example English localisation file (settings menu)
BEB2DD13552A07B1BAC2E1FC4DCD6837.dat [23.81 KiB]
Downloaded 12 times
File comment: Replaced strings
photo_2021-04-03_12-44-49.jpg [33.09 KiB]
Not downloaded yet
Top
   
PostPosted: Mon Apr 05, 2021 12:25 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 7
Good news, i managed to somewhat reverse engineer that format and now i can successfully extract strings (at least from file i sent previously)

Here's Python code that i used to extract file to ".po" file for translation
Code:
import struct, polib

locFile = open("test.dat", "rb")

if struct.unpack('cccc', locFile.read(4)) == (b'@', b'C', b'T', b'D'):
    _, = struct.unpack('h', locFile.read(2))  # Version?
    _, = struct.unpack('i', locFile.read(4))  # Padding

    unknown, = struct.unpack("i", locFile.read(4))
    locEntries, = struct.unpack("h", locFile.read(2))
    addrBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    hashBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    strBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    _, = struct.unpack("i", locFile.read(4))  # Padding

    print("Entries: " + str(locEntries))
    print("----------------------")
    print("Address block offset: " + str(hex(addrBlockOffset)))
    print("Hash Block Offset: " + str(hex(hashBlockOffset)))
    print("String block offset: " + str(hex(strBlockOffset)))
    print("----------------------")
   
    locFile.seek(addrBlockOffset)

    offsets = []

    for i in range(locEntries):
        locID, setID, = struct.unpack('HH', locFile.read(4))
        offset, unknown, = struct.unpack("hh", locFile.read(4))
        offsets.append(offset)

    locFile.seek(strBlockOffset)

    i = 0
    poFile = polib.POFile()

    for offset in offsets:
        locFile.seek(offset)
        lastChar = b""

        while lastChar != b"\0":
            lastChar = locFile.read(1)
            locFile.read(1)

        strLen = locFile.tell() - offset - 2
        text = u""

        if strLen != 0:
            locFile.seek(offset)

            rawStr = locFile.read(strLen)
            text += rawStr.decode('utf-16')

        print(str(i) + " - " + text)

        poFile.append(polib.POEntry(
            msgid=text,
            msgstr="",
            occurrences=[("test", str(i))]
        ))

        i += 1
   
    poFile.save("test.po")


Re-importing shouldn't be a problem if output file will be no-longer than original file, but as far as i can see archive itself is pretty "flexible"

Here's proof, that this scripts outputs valid .po files:
Attachment:
Zrzut ekranu 2021-04-05 142442.png [42.01 KiB]
Not downloaded yet


Top
   
PostPosted: Mon Apr 05, 2021 2:11 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 7
Okay, i finished both exporter and reimporter for locale files. It seems to work (i can freely modify texts inside settings menu), can someone help me identifying font files? And helping me replacing/adding characters?


Exporter:
Code:
import struct, polib

locFile = open("test.dat", "rb")

if struct.unpack('cccc', locFile.read(4)) == (b'@', b'C', b'T', b'D'):
    _, = struct.unpack('h', locFile.read(2))  # Version?
    _, = struct.unpack('i', locFile.read(4))  # Padding

    unknown, = struct.unpack("i", locFile.read(4))
    locEntries, = struct.unpack("h", locFile.read(2))
    addrBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    hashBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    strBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    _, = struct.unpack("i", locFile.read(4))  # Padding

    print("Entries: " + str(locEntries))
    print("----------------------")
    print("Address block offset: " + str(hex(addrBlockOffset)))
    print("Hash Block Offset: " + str(hex(hashBlockOffset)))
    print("String block offset: " + str(hex(strBlockOffset)))
    print("----------------------")
   
    locFile.seek(addrBlockOffset)

    offsets = []

    for i in range(locEntries):
        locID, setID, = struct.unpack('HH', locFile.read(4))
        offset, unknown, = struct.unpack("hh", locFile.read(4))
        offsets.append(offset)

    locFile.seek(strBlockOffset)

    i = 0
    poFile = polib.POFile()

    for offset in offsets:
        locFile.seek(offset)
        lastChar = b""

        while lastChar != b"\0":
            lastChar = locFile.read(1)
            locFile.read(1)

        strLen = locFile.tell() - offset - 2
        text = u""

        if strLen != 0:
            locFile.seek(offset)

            rawStr = locFile.read(strLen)
            text += rawStr.decode('utf-16')

        poFile.append(polib.POEntry(
            msgid=text,
            msgstr=""
        ))

        i += 1

    poFile.save("test.po")


Reimporter:
Code:
import polib, struct

origFile = open("test.dat", "rb")
newFile = open("BEB2DD13552A07B1BAC2E1FC4DCD6837.dat", "wb")

origFile.seek(0x18)  # strBlockOffset location
strBlockOffset, = struct.unpack("h", origFile.read(2))
origFile.seek(0)

translated = polib.pofile("test.po")
newFile.write(origFile.read(0x20))

offset = strBlockOffset

for string in translated:
    strLen = 0

    if string.msgstr == "" and string.msgid == "":
        strLen = len("\0".encode("utf-16-le"))
    elif string.msgstr == "":
        strLen = len(string.msgid.encode("utf-16-le")) + 2
    else:
        strLen = len(string.msgstr.encode("utf-16-le")) + 2

    newFile.write(origFile.read(4))
    newFile.write(struct.pack("h", offset))
    origFile.read(2)
    newFile.write(origFile.read(2))

    offset = offset + strLen

newFile.write(origFile.read(strBlockOffset - origFile.tell()))

for string in translated:
    if string.msgstr == "" and string.msgid == "":
        newFile.write("\0".encode("utf-16-le"))
    elif string.msgstr == "":
        newFile.write(string.msgid.encode("utf-16-le"))
        newFile.write("\0".encode("utf-16-le"))
    else:
        newFile.write(string.msgstr.encode("utf-16-le"))
        newFile.write("\0".encode("utf-16-le"))

origFile.seek(-4, 2)
newFile.write(origFile.read(4))
origFile.close()
newFile.close()


Top
   
PostPosted: Fri Apr 09, 2021 4:14 pm 

Joined: Thu Apr 01, 2021 4:32 pm
Posts: 2
GrzybDev, good afternoon.
Is this Python script used in quickbms to export and import files?
Thank you.


Top
   
PostPosted: Mon Apr 12, 2021 11:34 am 

Joined: Thu Feb 23, 2017 10:39 am
Posts: 88
Nice does this allow full unpack of the directories and access to the resources?


Top
   
PostPosted: Mon Apr 12, 2021 8:09 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 7
Code that i posted here was only proof of concept

I'm currently writing an tool, for unpacking and packing all archive files (and decompiling and compiling of locale files, i temporarily removed locale tool because "encrypted" archives contains format which is a little bit diffirent from not-encrypted ones, but it will be soon available)

Code: https://github.com/GrzybDev/KingdomHeartsTool

Currently this tool are able to unpack all encrypted and not encrypted archives /w compression etc., pack tool might work only on files which are originally not-encrypted and not-compressed (because i'm not saving some important data, and i'm not able to create byte-perfect archives... yet)

Anyway, if anyone really want to use that tool asap, even if pack feature is not complete here is example on how to use my tool:
Unpack archive:
Code:
python -m khtool -v -u "B:\Kingdom Hearts HD 1 5 and 2 5 ReMIX\Image\en\bbs_first.hed" -o output

Repack archive:
Code:
python -m khtool -v -p output/bbs_first.json -o output/bbs_first


Edit: Now i finished both unpack and pack tool, so you can freely repack all game archives (if you use both unpack and pack tool on the same set of data, pack tool will create byte-perfect archive)


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 9 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited