ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Fri Jun 22, 2018 7:33 am

All times are UTC




Post new topic  Reply to topic  [ 25 posts ]  Go to page 1 2 Next
Author Message
PostPosted: Fri Jan 12, 2018 12:38 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
Just a quick announcement about an alternative reimport mode implemented in QuickBMS 0.8.2a.

Now in the quickbms package you will find a new reimport2.bat file that activates this new mode.
REIMPORT2 is no longer limited by the size and compressed size of the reimported files, if the file is bigger it will be appended at the end of the archive.

The usage is the same of reimport.bat, the only difference for the user is that you will no longer see the usual "infamous" error:
Code:
Error: file "???"
       the reimport option acts as a reimporter and so you cannot reinsert a
       file if it's bigger than the original otherwise it will overwrite the
       rest of the archive or cannot be loaded correctly:

         new size: ? uncompressed
         old size: ? uncompressed

- do you want to skip this file? (y/N/force)


It doesn't mean that your edited archive will work "for sure" with your game, it just offers a way to bypass 2 important limits of the standard reimport mode:
  • size of the files
  • TOC information with some archives/games, in fact the offset, size and compressed size fields will be rewritten

Technical explanation:

Example of input archive
Code:
###112233################DATADATADATA#######
   | | |                 |
   | | |                 file
   | | size
   | compressed size
   offset
Standard reimport mode
Code:
###112233################DATADATADATA#######
                         |          |
                         | new file |
New reimport2 mode
Code:
###112233################DATADATADATA#######DATADATADATADATADATA
   | | |                                    |                  |
   | | |                                    | new bigger file! |
   | | new size
   | new compressed size
   new offset

As usual you will find all the info and limits (don't expect miracles) in section 3 of http://aluigi.org/papers/quickbms.txt

Feel free to post your positive and negative feedback about reimport2 here in this topic :)


Top
   
PostPosted: Fri Jan 12, 2018 1:42 pm 

Joined: Fri Nov 21, 2014 4:03 am
Posts: 328
Great works! Amazing! :o
:lol:


Top
   
PostPosted: Fri Jan 12, 2018 4:05 pm 

Joined: Fri Aug 05, 2016 6:17 pm
Posts: 46
thanks a lot Sensei :D


Top
   
PostPosted: Fri Jan 12, 2018 7:40 pm 
User avatar

Joined: Tue Oct 13, 2015 1:26 pm
Posts: 219
Amazing news!!! With this, QuickBMS will become the definitive game moding/hacking tool.
Great work master!

_________________
Tribo Gamer Brasil


Top
   
PostPosted: Sat Jan 13, 2018 7:08 am 

Joined: Mon Oct 06, 2014 6:41 pm
Posts: 327
Amazing !!! Great work. Thanks Aluigi :D


Top
   
PostPosted: Mon Jan 15, 2018 8:36 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
Not sure if it may be useful, but for the next version of quickbms I have in mind a sort of "install"/"uninstall" option that uses the REIMPORT2 mode.
Let's say you make a translation project that modifies 10 files of the archive and you want to distribute it easily and allow the user to apply it as a "patch" that he can remove if necessary.
Technically that would be easily possible by saving the original offset/size/zsize fields and appending all the new files (even those smaller than the originals) keeping the original files untouched, so it's easy to revert the modifications by restoring the original values and "cutting" the archive to its original size.

Anyway, a part from this idea, currently I'm mainly interested in what games work and don't work with reimport2, just for curiosity and "statistics".
Waiting your WORK / DOESN'T WORK list :)

*edit*
Right, the archives with a TOC located at the end of the archive (the "tail") can't work in reimport2 with files bigger than the originals since the TOC is no longer located at the end of the archive.
Just for statistics, In my collection of over 1800 scripts about 70 have the TOC at the end of the archive.


Top
   
PostPosted: Tue Jan 16, 2018 5:28 pm 
User avatar

Joined: Sun Dec 27, 2015 10:22 pm
Posts: 51
Awesome! So there's no longer a need for that "5GB quickbms" executable?


Top
   
PostPosted: Tue Jan 16, 2018 5:37 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
quickbms_4gb_files.exe is necessary for archives bigger than 4 Gb because it uses 64bit variables instead of 32bit.
It's also necessary in specific scripts that require 64bit operations.


Top
   
PostPosted: Thu Jan 18, 2018 3:55 am 
User avatar

Joined: Tue Oct 13, 2015 1:26 pm
Posts: 219
aluigi wrote:
Right, the archives with a TOC located at the end of the archive (the "tail") can't work in reimport2 with files bigger than the originals since the TOC is no longer located at the end of the archive.
Well, how about a parameter to set a offset at the end of file to reinsert the content? So, this way we can insert the mod file before the TOC or after the last content of the file.
aluigi wrote:
Anyway, a part from this idea, currently I'm mainly interested in what games work and don't work with reimport2, just for curiosity and "statistics".
Waiting your WORK / DOESN'T WORK list :)
Unreal Engine 4 PAK files have a TOC in the end of file, so should not work.

_________________
Tribo Gamer Brasil


Top
   
PostPosted: Thu Jan 18, 2018 4:01 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
Delutto wrote:
Well, how about a parameter to set a offset at the end of file to reinsert the content? So, this way we can insert the mod file before the TOC or after the last content of the file.

The only work-around I can see about the TOC-tails is simply doing reimport2 as-is now and then copying the whole TOC at the end of the file, but it will never work with the archives that write the offset of the TOC at the end (many):

goto -8
idstring "SIGN"
get TOC_OFFSET long
goto TOC_OFFSET

That would work only with archives that don't have a TOC offset, like those in which the starting of the TOC is calculated from the number of files or from the size of the TOC, for example: savepos OFFSET ; xmath OFFSET "OFFSET - (FILES * 8)" ; goto OFFSET.
And even in that lucky rare case there is still the problem that quickbms can't magically guess where the TOC starts and ends.

Long story short: no solution or work-around.

No problems for those archives with TOC at the end and TOC_OFFSET at the beginning since doesn't matter where the TOC and the files are located, that already works.

Ah in the "about 70" scripts I guessed as TOC-tail in my previous post I forgot to calculate those having the TOC offset at the beginning and those using "get OFFSET asize ; math OFFSET - ?" instead of "goto -?", so the number is a bit higher, probably still under 100 which is basically "nothing" compared to 1900 scripts.


Top
   
PostPosted: Sat Jan 20, 2018 2:16 am 

Joined: Mon Oct 06, 2014 3:30 am
Posts: 178
Nice! That's great!


Top
   
PostPosted: Mon Jan 22, 2018 8:33 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
In quickbms 0.8.3 I have started to add a feature that tells the user if the archive is invalid for reimport2, it will be improved over time because currently it simply checks if the scripts uses "goto -NUM" (TOC at the end).

I'm not sure if it's a good idea to zero the bytes where the original file is located in the archive, in theory yes because that space is no longer used and the archive will compress better if you want to distribute it.
I preferred to wait, let me know if you find something wrong in doing that.

I don't think I will implement the patch/unpatch option I mentioned some posts above because currently I have not enough statistics to know if it would be worth or not.


Top
   
PostPosted: Fri Jan 26, 2018 8:36 pm 

Joined: Fri Nov 21, 2014 4:03 am
Posts: 328
Test result - Doesn't work on Hitman 2016.


Top
   
PostPosted: Fri Jan 26, 2018 9:02 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
I guess the problem with hitman is caused by:
  • ZSIZE which is a result of various "&"
  • SIZE coming from a memory file
  • HASH may be checked by the game at runtime, but I don't know if it's the hash of name (probably) or file (I doubt)


Top
   
PostPosted: Mon Jan 29, 2018 12:49 pm 

Joined: Sat Jan 10, 2015 10:03 am
Posts: 49
i just tried REIMPORT2 with the evil within 2 but game still crashing... i think because game use ptr for offset and size and pkr for compressed file. any solution?


Top
   
PostPosted: Thu Feb 01, 2018 4:00 pm 

Joined: Fri Nov 21, 2014 4:03 am
Posts: 328
aluigi wrote:
I guess the problem with hitman is caused by:
  • ZSIZE which is a result of various "&"
  • SIZE coming from a memory file
  • HASH may be checked by the game at runtime, but I don't know if it's the hash of name (probably) or file (I doubt)


Thanks for your reply. :)


Top
   
PostPosted: Sat Feb 03, 2018 6:04 am 

Joined: Sun Nov 12, 2017 12:30 pm
Posts: 42
Excelent!!!


Top
   
PostPosted: Mon Feb 05, 2018 10:51 pm 
User avatar

Joined: Tue Oct 13, 2015 1:26 pm
Posts: 219
aluigi wrote:
Waiting your WORK / DOESN'T WORK list :)

REIMPORT 2 doesn't work with yoyogames.bms, more specifically with STRG(Text) file. (Or am I doing something wrong)
I think Slog is the problem(as descripted in section 3 of quickbms.txt), I tested some binary STRG dump with a following script:
Code:
get STRINGS long
for i = 0 < STRINGS
   get DUMMY long
next i
for i = 0 < STRINGS
   get STRING_SIZE long
   math STRING_SIZE + 1
   slog "Textos.txt" -1 STRING_SIZE
next i

Attachment:
File comment: STRG binary dump
STRG.7z [74.43 KiB]
Downloaded 43 times

_________________
Tribo Gamer Brasil


Top
   
PostPosted: Tue Feb 06, 2018 9:04 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8306
Offset -1 means that there is no absolute offset, so the strings are stored sequentially and therefore it's not possible to store a string longer than the original


Top
   
PostPosted: Sat Feb 24, 2018 11:45 am 
User avatar

Joined: Wed Jul 19, 2017 5:04 am
Posts: 60
in Force mod, archive will work with a new size or not? (for those script can be repack(inject) a file in archive)

_________________
Be Good, Even in distressing or defamation and chill, Even Everyone's Called You a Bad Guy Or Lier and Mad


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 25 posts ]  Go to page 1 2 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited