ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Mon May 17, 2021 9:17 am

All times are UTC




Post new topic  Reply to topic  [ 22 posts ]  Go to page 1 2 Next
Author Message
PostPosted: Wed Mar 31, 2021 9:35 pm 

Joined: Thu Nov 26, 2020 2:13 pm
Posts: 29
Code:
# KINGDOM HEARTS HD 1.5 + 2.5 ReMIX (Microsoft Windows)
# KINGDOM HEARTS HD 2.8 Final Chapter Prologue (Microsoft Windows)

math is_hed = 0
math is_pkg = 0

get ext extension
if ext == "hed"
   math is_pkg = 1
   callfunction handle_hed_pkg 1
elif ext == "pkg"
   math is_hed = 1
   callfunction handle_hed_pkg 1
endif

startfunction handle_hed_pkg
   open FDDE "hed" is_hed
   open FDDE "pkg" is_pkg
   get hed_size asize is_hed
   xmath hed_files "hed_size / 0x20"
   for i = 0 < hed_files
      set name string ""
      getdstring hash 0x10 is_hed
      string hash_in_str b hash
      putvarchr hash_in_str 32 0
      get file_offset longlong is_hed
      get file_size1 long is_hed
      get file_size2 long is_hed
      set name string hash_in_str
      goto file_offset is_pkg
      get file_size3 long is_pkg
      get sub_entries long is_pkg
      get compressed_file_size1 long is_pkg
      get file_info3 long is_pkg
      if sub_entries == 0
         string name + "."
         math file_offset + 16
         if compressed_file_size1 <= 0xfffffff0
            log name file_offset compressed_file_size1 is_pkg
         else
            log name file_offset file_size3 is_pkg
         endif
      else
         string name1 p "%s/init." name
         xmath helper "16 + (sub_entries * 0x30)"
         math file_offset + helper
         if compressed_file_size1 <= 0xfffffff0
            log name1 file_offset compressed_file_size1 is_pkg
            math file_offset + compressed_file_size1
         else
            log name1 file_offset file_size3 is_pkg
            math file_offset + file_size3
         endif
         for j = 0 < sub_entries
            string name2 p "%s/%u." name j
            getdstring sbe_name 0x20 is_pkg
            get sbe1 long is_pkg
            get sbe2 long is_pkg
            get sbe3 long is_pkg
            get sbe4 long is_pkg
            if sbe4 <= 0xfffffff0
               log name2 file_offset sbe4 is_pkg
               math file_offset + sbe4
            else
               log name2 file_offset sbe3 is_pkg
               xmath sbe3_1 "sbe3 && 0x10"
               math file_offset + sbe3_1
            endif
         next j
      endif
   next i
endfunction
this script can only extract files as-is, as a consequence it cannot handle any decompression stuff whatsoever. this is why you get some garbled files.


Top
   
PostPosted: Thu Apr 01, 2021 4:39 pm 

Joined: Thu Apr 01, 2021 4:32 pm
Posts: 2
Hello tbmq008.
I have a project to translate the game into Portuguese-BR and I am looking for tips on how to extract the texts from the game.
Use the bms with the script above with the files "SettingMenu.hed" and "SettingMenu.pkg".
When extracting, I have the files ".sed", ".l2d" and ".dat".
Any ideas on how to extract texts from menus and dialogues?

Thanks.


Top
   
PostPosted: Thu Apr 01, 2021 5:24 pm 

Joined: Thu Nov 26, 2020 2:13 pm
Posts: 29
".sed" files are simply sound data.
as for the rest i have no idea, i went in and wrote the script in "blind mode".


Top
   
PostPosted: Sat Apr 03, 2021 11:19 am 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
As far as i can see all locale files starts with "@CTD", then some metadata and then UTF-16 encoded strings.
Replacing strings should be easy if someone figures out that metadata before strings :)

I've uploaded file where you can see results of "translating" this file to blah language :D (it's the settings menu in launcher),

Image


Attachments:
File comment: Example English localisation file (settings menu)
BEB2DD13552A07B1BAC2E1FC4DCD6837.dat [23.81 KiB]
Downloaded 25 times
File comment: Replaced strings
photo_2021-04-03_12-44-49.jpg [33.09 KiB]
Not downloaded yet
Top
   
PostPosted: Mon Apr 05, 2021 12:25 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
Good news, i managed to somewhat reverse engineer that format and now i can successfully extract strings (at least from file i sent previously)

Here's Python code that i used to extract file to ".po" file for translation
Code:
import struct, polib

locFile = open("test.dat", "rb")

if struct.unpack('cccc', locFile.read(4)) == (b'@', b'C', b'T', b'D'):
    _, = struct.unpack('h', locFile.read(2))  # Version?
    _, = struct.unpack('i', locFile.read(4))  # Padding

    unknown, = struct.unpack("i", locFile.read(4))
    locEntries, = struct.unpack("h", locFile.read(2))
    addrBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    hashBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    strBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    _, = struct.unpack("i", locFile.read(4))  # Padding

    print("Entries: " + str(locEntries))
    print("----------------------")
    print("Address block offset: " + str(hex(addrBlockOffset)))
    print("Hash Block Offset: " + str(hex(hashBlockOffset)))
    print("String block offset: " + str(hex(strBlockOffset)))
    print("----------------------")
   
    locFile.seek(addrBlockOffset)

    offsets = []

    for i in range(locEntries):
        locID, setID, = struct.unpack('HH', locFile.read(4))
        offset, unknown, = struct.unpack("hh", locFile.read(4))
        offsets.append(offset)

    locFile.seek(strBlockOffset)

    i = 0
    poFile = polib.POFile()

    for offset in offsets:
        locFile.seek(offset)
        lastChar = b""

        while lastChar != b"\0":
            lastChar = locFile.read(1)
            locFile.read(1)

        strLen = locFile.tell() - offset - 2
        text = u""

        if strLen != 0:
            locFile.seek(offset)

            rawStr = locFile.read(strLen)
            text += rawStr.decode('utf-16')

        print(str(i) + " - " + text)

        poFile.append(polib.POEntry(
            msgid=text,
            msgstr="",
            occurrences=[("test", str(i))]
        ))

        i += 1
   
    poFile.save("test.po")


Re-importing shouldn't be a problem if output file will be no-longer than original file, but as far as i can see archive itself is pretty "flexible"

Here's proof, that this scripts outputs valid .po files:
Attachment:
Zrzut ekranu 2021-04-05 142442.png [42.01 KiB]
Not downloaded yet


Top
   
PostPosted: Mon Apr 05, 2021 2:11 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
Okay, i finished both exporter and reimporter for locale files. It seems to work (i can freely modify texts inside settings menu), can someone help me identifying font files? And helping me replacing/adding characters?


Exporter:
Code:
import struct, polib

locFile = open("test.dat", "rb")

if struct.unpack('cccc', locFile.read(4)) == (b'@', b'C', b'T', b'D'):
    _, = struct.unpack('h', locFile.read(2))  # Version?
    _, = struct.unpack('i', locFile.read(4))  # Padding

    unknown, = struct.unpack("i", locFile.read(4))
    locEntries, = struct.unpack("h", locFile.read(2))
    addrBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    hashBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    strBlockOffset, _, = struct.unpack("hh", locFile.read(4))
    _, = struct.unpack("i", locFile.read(4))  # Padding

    print("Entries: " + str(locEntries))
    print("----------------------")
    print("Address block offset: " + str(hex(addrBlockOffset)))
    print("Hash Block Offset: " + str(hex(hashBlockOffset)))
    print("String block offset: " + str(hex(strBlockOffset)))
    print("----------------------")
   
    locFile.seek(addrBlockOffset)

    offsets = []

    for i in range(locEntries):
        locID, setID, = struct.unpack('HH', locFile.read(4))
        offset, unknown, = struct.unpack("hh", locFile.read(4))
        offsets.append(offset)

    locFile.seek(strBlockOffset)

    i = 0
    poFile = polib.POFile()

    for offset in offsets:
        locFile.seek(offset)
        lastChar = b""

        while lastChar != b"\0":
            lastChar = locFile.read(1)
            locFile.read(1)

        strLen = locFile.tell() - offset - 2
        text = u""

        if strLen != 0:
            locFile.seek(offset)

            rawStr = locFile.read(strLen)
            text += rawStr.decode('utf-16')

        poFile.append(polib.POEntry(
            msgid=text,
            msgstr=""
        ))

        i += 1

    poFile.save("test.po")


Reimporter:
Code:
import polib, struct

origFile = open("test.dat", "rb")
newFile = open("BEB2DD13552A07B1BAC2E1FC4DCD6837.dat", "wb")

origFile.seek(0x18)  # strBlockOffset location
strBlockOffset, = struct.unpack("h", origFile.read(2))
origFile.seek(0)

translated = polib.pofile("test.po")
newFile.write(origFile.read(0x20))

offset = strBlockOffset

for string in translated:
    strLen = 0

    if string.msgstr == "" and string.msgid == "":
        strLen = len("\0".encode("utf-16-le"))
    elif string.msgstr == "":
        strLen = len(string.msgid.encode("utf-16-le")) + 2
    else:
        strLen = len(string.msgstr.encode("utf-16-le")) + 2

    newFile.write(origFile.read(4))
    newFile.write(struct.pack("h", offset))
    origFile.read(2)
    newFile.write(origFile.read(2))

    offset = offset + strLen

newFile.write(origFile.read(strBlockOffset - origFile.tell()))

for string in translated:
    if string.msgstr == "" and string.msgid == "":
        newFile.write("\0".encode("utf-16-le"))
    elif string.msgstr == "":
        newFile.write(string.msgid.encode("utf-16-le"))
        newFile.write("\0".encode("utf-16-le"))
    else:
        newFile.write(string.msgstr.encode("utf-16-le"))
        newFile.write("\0".encode("utf-16-le"))

origFile.seek(-4, 2)
newFile.write(origFile.read(4))
origFile.close()
newFile.close()


Top
   
PostPosted: Fri Apr 09, 2021 4:14 pm 

Joined: Thu Apr 01, 2021 4:32 pm
Posts: 2
GrzybDev, good afternoon.
Is this Python script used in quickbms to export and import files?
Thank you.


Top
   
PostPosted: Mon Apr 12, 2021 11:34 am 

Joined: Thu Feb 23, 2017 10:39 am
Posts: 96
Nice does this allow full unpack of the directories and access to the resources?


Top
   
PostPosted: Mon Apr 12, 2021 8:09 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
Code that i posted here was only proof of concept

I'm currently writing an tool, for unpacking and packing all archive files (and decompiling and compiling of locale files, i temporarily removed locale tool because "encrypted" archives contains format which is a little bit diffirent from not-encrypted ones, but it will be soon available)

Code: https://github.com/GrzybDev/KingdomHeartsTool

Currently this tool are able to unpack all encrypted and not encrypted archives /w compression etc., pack tool might work only on files which are originally not-encrypted and not-compressed (because i'm not saving some important data, and i'm not able to create byte-perfect archives... yet)

Anyway, if anyone really want to use that tool asap, even if pack feature is not complete here is example on how to use my tool:
Unpack archive:
Code:
python -m khtool -v -u "B:\Kingdom Hearts HD 1 5 and 2 5 ReMIX\Image\en\bbs_first.hed" -o output

Repack archive:
Code:
python -m khtool -v -p output/bbs_first.json -o output/bbs_first


Edit: Now i finished both unpack and pack tool, so you can freely repack all game archives (if you use both unpack and pack tool on the same set of data, pack tool will create byte-perfect archive)


Top
   
PostPosted: Fri Apr 23, 2021 7:27 pm 

Joined: Fri Apr 16, 2021 7:56 pm
Posts: 7
I am trying to translate the menu, the menu file is Image / SettingMenu.hed I used the following method:
python -m khtool -v -u "C:\Games\Kingdom Hearts HD 1 5 and 2 5 ReMIX\Image\SettingMenu.hed" -o output\

Then:
python -m khtool -v -l "output/SettingMenu/*.ctd" -o traduzir/
I translated the file BEB2DD13552A07B1BAC2E1FC4DCD6837.po because this file is from the menu in English. I translated:

msgid "Quit Game"
to:
msgid "Sair do Jogo"

Then:
python -m khtool -v -l "traduzir/output/SettingMenu/*.po" -o final/

I copied the files from the final folder and replaced the one in the output / SettingMenu folder and executed the command:
python -m khtool -v -p output/SettingMenu.json -o output/SettingMenu

I copied the files SettingMenu.hed and SettingMenu.pkg to the Image folder, but it didn't work


Attachments:
Traducao2.png [475.66 KiB]
Not downloaded yet
Traducao.png [38.94 KiB]
Not downloaded yet
Top
   
PostPosted: Sat Apr 24, 2021 6:22 am 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
There's small problem with offset calculating, will fix it asap


Top
   
PostPosted: Tue Apr 27, 2021 7:57 pm 

Joined: Fri Apr 16, 2021 7:56 pm
Posts: 7
When you fix it, will you notify me here on the forum or will I see it for the last commit there in git?


Top
   
PostPosted: Thu Apr 29, 2021 6:41 am 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
The issue is now fixed, my bad because of old code

I see that you translated msgid entry...
Dude, you never translated anything, before you start READ THE MANUAL https://www.gnu.org/software/gettext/ma ... Files.html


Top
   
PostPosted: Thu Apr 29, 2021 12:17 pm 

Joined: Fri Apr 16, 2021 7:56 pm
Posts: 7
Now it's working. I managed to translate some parts of the menu and it worked, thanks.


Attachments:
traducao.jpg [163.85 KiB]
Not downloaded yet
Top
   
PostPosted: Thu Apr 29, 2021 1:26 pm 

Joined: Fri Apr 16, 2021 7:56 pm
Posts: 7
I did an accentual translation, and it didn't work, how could this be done or not?
The word is: Exibição


Attachments:
traducao.png [478.94 KiB]
Not downloaded yet
Top
   
PostPosted: Thu Apr 29, 2021 1:34 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
There's no problem with my tool, it's a problem because your language uses characters that this game do not understand (hence ? instead of chars)
You will need to replace already existing characters in font file, with characters you want.


Top
   
PostPosted: Thu Apr 29, 2021 2:00 pm 

Joined: Fri Apr 16, 2021 7:56 pm
Posts: 7
Could you explain to me how to do this? I don't know how to change to accept Brazilian characters.

It is strange not to accept the word ã, but the word õ accepts


Top
   
PostPosted: Thu Apr 29, 2021 2:45 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
It is not a question for me, i only made a tool which allows to compile and extract strings from game files
You need to do your own research on how to modify font files, because i'm not interested in graphic files (yet)

My team provided me list of characters from font, so it is possible:
Image


Attachments:
Zrzut ekranu 2021-04-29 164424.png [262.72 KiB]
Not downloaded yet
Top
   
PostPosted: Thu Apr 29, 2021 2:48 pm 

Joined: Fri Apr 16, 2021 7:56 pm
Posts: 7
Ok I understand, I just want to know which file to change the font?


Top
   
PostPosted: Thu Apr 29, 2021 2:52 pm 

Joined: Fri Apr 26, 2019 10:18 am
Posts: 22
I don't know, as i stated i only care about text files. And it's obviously not one font, but at least 20 (from what i can see)

They can be anywhere, maybe they are in .i2d files, maybe .tm2


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 22 posts ]  Go to page 1 2 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited