ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sun Jan 20, 2019 8:42 am

All times are UTC




Post new topic  Reply to topic  [ 385 posts ]  Go to page Previous 116 17 18 19 20 Next
Author Message
PostPosted: Sun Sep 23, 2018 1:11 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
The fake compressors are a huge success.
I already implemented most of those that were missing in my scripts and they all work:
Code:
DRV3_SRD_COMPRESS
YAZ0_COMPRESS
BIZARRE_COMPRESS
BIZARRE_SKIP_COMPRESS
BLACKDESERT_COMPRESS
DR12AE_COMPRESS
EA_COMP_COMPRESS
LBALZSS_COMPRESS
MOHLZSS_COMPRESS
MOHRLE_COMPRESS
NISLZS_COMPRESS
QCMP1_COMPRESS
RFPK_COMPRESS
RLEW_COMPRESS
SAINT_SEYA_COMPRESS
SHREK_COMPRESS
SLZ_01_COMPRESS
SLZ_02_COMPRESS
SLZ_03_COMPRESS
UCLPACK_COMPRESS
SEGA_LZS2_COMPRESS
WOLF_COMPRESS
YAKUZA_COMPRESS
YKCMP_COMPRESS
Implementing them is really a joke requiring one minute if you have the source code of the decompression algorithm and no bit-operations are involved.


Top
   
PostPosted: Sun Sep 30, 2018 12:47 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 82
looks like QuickBMS has issue with single files in reimport2
can you just add a tag to replace raw file (not re-overwrite OR write in next data)
i know its a problem in a single file without OFFSET, but a command-option must disable this error and replace raw file, not write. just for single files that no offset are located

if you do that, then we can even make scrip for text and string files!

i really cant reimport2 such files!


Last edited by Shokoniraya on Thu Nov 15, 2018 5:34 am, edited 2 times in total.

Top
   
PostPosted: Wed Oct 03, 2018 10:58 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
In that case you should use reimport.bat and then select "force" when quickbms asks you if you want to continue due to your file being bigger than the original.

There is no issue or bug, reimport.bat and reimport2.bat work exactly as they are meant to do :)


Top
   
PostPosted: Thu Oct 04, 2018 11:15 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 82
aluigi wrote:
In that case you should use reimport.bat and then select "force" when quickbms asks you if you want to continue due to your file being bigger than the original.

There is no issue or bug, reimport.bat and reimport2.bat work exactly as they are meant to do :)


i know about force mode, quick bms cant updata file size in force mode and reimport2 cant import bigger file in whit this script
my QuickBMS vesion; 0.9.0 (2018-17 jun)


Last edited by Shokoniraya on Thu Nov 15, 2018 5:34 am, edited 1 time in total.

Top
   
PostPosted: Thu Oct 04, 2018 1:30 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
You don't need quickbms to do something like that and in any case you are doing it in the wrong way because it's not just a single file that you can put back ignoring all the data that was available after it and the other fields referring to sizes and offsets.
You have at least 2 size fields at offset 0x90 and 0x94 and the data from offset 0x351b is NOT the ogg file, it's part of the archive format/container that you are going to screw by overwriting them.
It's extremely important to never overwrite data that doesn't belong to the extracted file and why reimport and reimport2 work in that way.


Top
   
PostPosted: Fri Oct 05, 2018 7:09 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 82
aluigi wrote:
You don't need quickbms to do something like that and in any case you are doing it in the wrong way because it's not just a single file that you can put back ignoring all the data that was available after it and the other fields referring to sizes and offsets.
You have at least 2 size fields at offset 0x90 and 0x94 and the data from offset 0x351b is NOT the ogg file, it's part of the archive format/container that you are going to screw by overwriting them.
It's extremely important to never overwrite data that doesn't belong to the extracted file and why reimport and reimport2 work in that way.



Aluigi, then READ THIS :D

in this case, plase read this note carefully

i want to say some things, real and important. im gonna tell you some Property that Must add into QuickBMS.exe

1_Importing In Bigger Size;
as you can see, quick bms is verry good for anyone, who wants starting to editing files and archive. Admittedly, need To Improt Bigger Size in Files
im not tlking about reimport2 for set new offset and new size! im talking about memory_size or those files without offset!

2_Multi Size:
in my Previous post, my file had multi size,
i cant see any tag in QuickBMS.exe for update both size.
some archve has Archive_Size, like UMD (Unreal Engine 2.5) and for import2, Archive_size must turn to read base size!
also some archives has decompressed_size and compressed_size, and we cant do it with Quick bms, and thats not well for someone like me that cant make a new tool or dont know any other programing language Like C# or C++ and even C.
why QuickBMS? (for any other user who wants do the same without learning a real programming language. [Aluigi Auriemma]) remember? that was some good words from you

3-Single_Raw
i read some post from some poeple, like (SIMS 4 STBL). if you adding a tag to disable write and do replace Raw_File(in this case, user cant import one file, that means user must extract all and import all, not just import some file[for update all size]. and you must do it in reimport2, not force mode. or maybe, should add a command-option to enable force mode in reimport2 mode, that would be awsome and its a big Process for quickbms! [just for text files and single files])

4-log file
i wish i could had a log file to read and keep my history of quickbms useage, its useul for some guys like me.


Last edited by Shokoniraya on Thu Nov 15, 2018 5:35 am, edited 1 time in total.

Top
   
PostPosted: Fri Oct 05, 2018 8:15 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
1) unfortunately quickbms is not a magician and can't store 100 bytes in a place for 99 bytes. The only way to do that is the presence of an offset field that can be replaced with a new one pointing to the new data.

2) compressed and uncompressed sizes are fully supported in reimport2 when the Clog command is invoked, usually you have a compression flag in the archives that tells you if a file is compressed and I bet this is probably the case. It's up to you to understand the format and figure out what are the flag, compressed and uncompresse size fields.

Just for curiosity (not related to this case so ignore this phrase once avoiding confusion) there is only one rare case in which reimport2 can't work properly and it's when there is no compression flag, for example just a "if SIZE == ZSIZE ..." check but even in that case it's possible to make the script compatible by using comtype copy and clog instead of log.

3) honestly I have not understood but it looks something related to question 1 or you forgot to remove the non-edited files from the folder, quickbms reimports only the available files. While in extraction it's possible to filter the extracted files based on their names and extensions so if you have 1000000 files and only 10 jpg images you want to edit, it's better to use the -f "{}.jpg" option avoiding to extract 999990 useless files.

4) it depends by what type of log you need.
Launching quickbms with the -V option returns tons of useful technical information about the input data.
The -v option is rarely useful and it's related to the commands of the script.
They must be used from command-line maybe with the "less" cygwin tool (suggested) or with a simple redirect to file:
Code:
quickbms.exe -V -0 script .bms input_file.dat > log.txt
quickbms.exe -V -0 script .bms input_file.dat | less


Top
   
PostPosted: Fri Oct 05, 2018 10:00 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 82
i totally get it, aluigi! thank you for explain! :)
and i just want to tell you: can you add force mode in reimport2?
because in reimport1 force, file size cant update...
half of problem solved in force mode, but if you add force mode in reimport2 too, i can solve that!


Last edited by Shokoniraya on Thu Nov 15, 2018 5:35 am, edited 1 time in total.

Top
   
PostPosted: Fri Oct 05, 2018 4:22 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
reimport2 can't have a force mode because if the file is bigger it's automatically added at the end (without any questions to the user) and its offset/size fields rewritten.

I guess that what you want is a way for automatically replacing the offset/size fields and writing the file without caring what's the format, sort of "mix" of reimport+reimport2.
In that case it's better if you write a script for specifically doing that (not easy since it's not the job of quickbms) or just do it manually with a hex editor.

reimport.bat does not replace the original size field for compatibility with any format and for compatibility with the original behaviour of such feature: overwriting just the bytes of the file and nothing else.


Top
   
PostPosted: Fri Oct 05, 2018 7:44 pm 

Joined: Sun Aug 10, 2014 12:49 pm
Posts: 272
Shokoniraya wrote:
i totally get it, aluigi! thank you for explain! :)
and i just want to tell you: can you add force mode in reimport2?
because in reimport1 force, file size cant update...
half of problem solved in force mode, but if you add force mode in reimport2 too, i can solve that!



I dont think you understand the reimport2 method. There is no such a force needed neither necessary as aluigi told you, when file is bigger its automatically go to end of the archive and in data table rewrite is size and offset, etc or other props needed. This method can't be used in same cases of segmented archives i guess..


Top
   
PostPosted: Sat Oct 06, 2018 5:18 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 82
michalss wrote:
I dont think you understand the reimport2 method. There is no such a force needed neither necessary as aluigi told you, when file is bigger its automatically go to end of the archive and in data table rewrite is size and offset, etc or other props needed. This method can't be used in same cases of segmented archives i guess..


dear michalss: reimport2 means; set new offset and size and i totaly know that
and force mode means replace raw file

but we cant do it on single files white reimport2!
need a command option to replace raw files like force mode, not write, just like text files! and just adding a tag for do this thing...

go to end of the archive and in data table rewrite is size and offset: and set old data as "00 00 00 00", i know that.


Last edited by Shokoniraya on Thu Nov 15, 2018 5:35 am, edited 1 time in total.

Top
   
PostPosted: Sat Oct 06, 2018 8:24 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
What's probably not clear to many users is that the "force" option in reimport.bat is BAD and should be never used, NEVER.

It's something that corrupts the data and the only reason why it exists is that I preferred to give people a choice in case they really know what they are doing and that the edited archive will not work the 99.9% of times.
Please don't force me to remove that useless option from quickbms :)

Long story short: if you need to use the "force" option it means you are doing something completely wrong and you will receive no support.

That's the clear position about quickbms, but in your case, as already explained, you don't need quickbms to do something simple like that :)
Use a hex editor and do the job in one minute.


Top
   
PostPosted: Mon Oct 08, 2018 9:20 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
I would like to introduce a better way to use the reimport and reimport2 modes by performing some additional checks.

1)
If you use reimport.bat for reimporting 1000 files and one of them is bigger than the original then the reimporting will be interrupted.
Possible solution:
- ask the user if he wants to retry (y/N/r/force) so he can edit the file and retry without running the process again

2)
If you have extracted 100000 files and have edited only one of them, quickbms will try to reimport all the 100000 files... it may be a problem for modders because deleting the non-edited files before reimporting may be a difficult (boring) process in some games.
Possible solution:
- calculate an hash (sha1 or sha256) on the archived file and an hash of the one on disk, compare them and reimport the file only if it's different. obviously if the size is different there is no need to make the hash.
While this solution seems nice, it has the disadvantage that basically quickbms needs to read (and decompress, just a full extraction without disk writing) the input file and compute the hash spending time and resources, multiply it by two since the same is done on the file on disk too.
sha1 is faster than sha256 and in this non-security context it's probably the best choice since no collisions are expected while editing a file.
I may implement this feature as an option for quickbms but I don't know if I have to add it to reimport.bat and reimport2.bat


Top
   
PostPosted: Tue Oct 09, 2018 3:31 am 
User avatar

Joined: Wed Jul 19, 2017 5:04 am
Posts: 184
Shokoniraya wrote:
michalss wrote:
I dont think you understand the reimport2 method. There is no such a force needed neither necessary as aluigi told you, when file is bigger its automatically go to end of the archive and in data table rewrite is size and offset, etc or other props needed. This method can't be used in same cases of segmented archives i guess..


dear michalss: reimport2 means; set new offset and size and i totaly know that
and force mode means replace raw file

but we cant do it on single files white reimport2!
need a command option to replace raw files like force mode, not write, just like text files! and just adding a tag for do this thing...

go to end of the archive and in data table rewrite is size and offset: and set old data as "00 00 00 00", i know that.


yeah, right. but you can't do it with QuickBMS
Try to learn a language for making your own Tools

QuickBMS can't help in such thing: Text Files, Single Files, converting an File to another File Like PNG to tex. replace raw_data

_________________
Be Good, Even in distressing or defamation and chill, Even Everyone's Called You a Bad Guy Or Lier and Mad


Top
   
PostPosted: Tue Oct 30, 2018 7:07 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
quickbms 0.9.1 will use the Capstone library as disassembler and will support every architecture supported by the library, so not just x86.


Top
   
PostPosted: Wed Oct 31, 2018 4:27 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
reimport2 will work also with archives having the TOC at the end, maybe not 100% of times but most of them :)


Top
   
PostPosted: Thu Nov 01, 2018 9:31 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
I still have no release date for quickbms 0.9.1 but at least I have finished to write the (temporary) changelog (the list of things fixed and added).
Even today I have fixed some bugs that have been casually found during a test.
I have also updated some libraries.

The only priority at the moment is beta testing for spotting new issues.
Hopefully next week I will have good news :)


Top
   
PostPosted: Fri Nov 02, 2018 6:12 am 

Joined: Sun Aug 10, 2014 12:49 pm
Posts: 272
aluigi wrote:
I still have no release date for quickbms 0.9.1 but at least I have finished to write the (temporary) changelog (the list of things fixed and added).
Even today I have fixed some bugs that have been casually found during a test.
I have also updated some libraries.

The only priority at the moment is beta testing for spotting new issues.
Hopefully next week I will have good news :)



Are you also planning release fixed DDL with compress suport pls ? Waiting for this . :) thx


Top
   
PostPosted: Fri Nov 02, 2018 8:47 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9654
Sure


Top
   
PostPosted: Sun Nov 04, 2018 6:37 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 82
GHOST DEAD wrote:
yeah, right. but you can't do it with QuickBMS
Try to learn a language for making your own Tools

QuickBMS can't help in such thing: Text Files, Single Files, converting an File to another File Like PNG to tex. replace raw_data


i saw your posts. you had same problem, right? how did you solve it then? i cant do it

aluigi wrote:
Sure

really? gonna be Great! even with text files? gonna possible?
cant wait to make new scripts with Good Stuff!


Last edited by Shokoniraya on Thu Nov 15, 2018 5:36 am, edited 1 time in total.

Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 385 posts ]  Go to page Previous 116 17 18 19 20 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited