ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sat Nov 17, 2018 5:02 pm

All times are UTC




Post new topic  Reply to topic  [ 374 posts ]  Go to page Previous 115 16 17 18 19 Next
Author Message
PostPosted: Sun Sep 23, 2018 1:11 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
The fake compressors are a huge success.
I already implemented most of those that were missing in my scripts and they all work:
Code:
DRV3_SRD_COMPRESS
YAZ0_COMPRESS
BIZARRE_COMPRESS
BIZARRE_SKIP_COMPRESS
BLACKDESERT_COMPRESS
DR12AE_COMPRESS
EA_COMP_COMPRESS
LBALZSS_COMPRESS
MOHLZSS_COMPRESS
MOHRLE_COMPRESS
NISLZS_COMPRESS
QCMP1_COMPRESS
RFPK_COMPRESS
RLEW_COMPRESS
SAINT_SEYA_COMPRESS
SHREK_COMPRESS
SLZ_01_COMPRESS
SLZ_02_COMPRESS
SLZ_03_COMPRESS
UCLPACK_COMPRESS
SEGA_LZS2_COMPRESS
WOLF_COMPRESS
YAKUZA_COMPRESS
YKCMP_COMPRESS
Implementing them is really a joke requiring one minute if you have the source code of the decompression algorithm and no bit-operations are involved.


Top
   
PostPosted: Sun Sep 30, 2018 12:47 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 55
Removed


Last edited by Shokoniraya on Thu Nov 15, 2018 5:34 am, edited 2 times in total.

Top
   
PostPosted: Wed Oct 03, 2018 10:58 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
In that case you should use reimport.bat and then select "force" when quickbms asks you if you want to continue due to your file being bigger than the original.

There is no issue or bug, reimport.bat and reimport2.bat work exactly as they are meant to do :)


Top
   
PostPosted: Thu Oct 04, 2018 11:15 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 55
Removed


Last edited by Shokoniraya on Thu Nov 15, 2018 5:34 am, edited 1 time in total.

Top
   
PostPosted: Thu Oct 04, 2018 1:30 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
You don't need quickbms to do something like that and in any case you are doing it in the wrong way because it's not just a single file that you can put back ignoring all the data that was available after it and the other fields referring to sizes and offsets.
You have at least 2 size fields at offset 0x90 and 0x94 and the data from offset 0x351b is NOT the ogg file, it's part of the archive format/container that you are going to screw by overwriting them.
It's extremely important to never overwrite data that doesn't belong to the extracted file and why reimport and reimport2 work in that way.


Top
   
PostPosted: Fri Oct 05, 2018 7:09 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 55
Removed


Last edited by Shokoniraya on Thu Nov 15, 2018 5:35 am, edited 1 time in total.

Top
   
PostPosted: Fri Oct 05, 2018 8:15 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
1) unfortunately quickbms is not a magician and can't store 100 bytes in a place for 99 bytes. The only way to do that is the presence of an offset field that can be replaced with a new one pointing to the new data.

2) compressed and uncompressed sizes are fully supported in reimport2 when the Clog command is invoked, usually you have a compression flag in the archives that tells you if a file is compressed and I bet this is probably the case. It's up to you to understand the format and figure out what are the flag, compressed and uncompresse size fields.

Just for curiosity (not related to this case so ignore this phrase once avoiding confusion) there is only one rare case in which reimport2 can't work properly and it's when there is no compression flag, for example just a "if SIZE == ZSIZE ..." check but even in that case it's possible to make the script compatible by using comtype copy and clog instead of log.

3) honestly I have not understood but it looks something related to question 1 or you forgot to remove the non-edited files from the folder, quickbms reimports only the available files. While in extraction it's possible to filter the extracted files based on their names and extensions so if you have 1000000 files and only 10 jpg images you want to edit, it's better to use the -f "{}.jpg" option avoiding to extract 999990 useless files.

4) it depends by what type of log you need.
Launching quickbms with the -V option returns tons of useful technical information about the input data.
The -v option is rarely useful and it's related to the commands of the script.
They must be used from command-line maybe with the "less" cygwin tool (suggested) or with a simple redirect to file:
Code:
quickbms.exe -V -0 script .bms input_file.dat > log.txt
quickbms.exe -V -0 script .bms input_file.dat | less


Top
   
PostPosted: Fri Oct 05, 2018 10:00 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 55
Removed


Last edited by Shokoniraya on Thu Nov 15, 2018 5:35 am, edited 1 time in total.

Top
   
PostPosted: Fri Oct 05, 2018 4:22 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
reimport2 can't have a force mode because if the file is bigger it's automatically added at the end (without any questions to the user) and its offset/size fields rewritten.

I guess that what you want is a way for automatically replacing the offset/size fields and writing the file without caring what's the format, sort of "mix" of reimport+reimport2.
In that case it's better if you write a script for specifically doing that (not easy since it's not the job of quickbms) or just do it manually with a hex editor.

reimport.bat does not replace the original size field for compatibility with any format and for compatibility with the original behaviour of such feature: overwriting just the bytes of the file and nothing else.


Top
   
PostPosted: Fri Oct 05, 2018 7:44 pm 

Joined: Sun Aug 10, 2014 12:49 pm
Posts: 271
Shokoniraya wrote:
i totally get it, aluigi! thank you for explain! :)
and i just want to tell you: can you add force mode in reimport2?
because in reimport1 force, file size cant update...
half of problem solved in force mode, but if you add force mode in reimport2 too, i can solve that!



I dont think you understand the reimport2 method. There is no such a force needed neither necessary as aluigi told you, when file is bigger its automatically go to end of the archive and in data table rewrite is size and offset, etc or other props needed. This method can't be used in same cases of segmented archives i guess..


Top
   
PostPosted: Sat Oct 06, 2018 5:18 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 55
Removed


Last edited by Shokoniraya on Thu Nov 15, 2018 5:35 am, edited 1 time in total.

Top
   
PostPosted: Sat Oct 06, 2018 8:24 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
What's probably not clear to many users is that the "force" option in reimport.bat is BAD and should be never used, NEVER.

It's something that corrupts the data and the only reason why it exists is that I preferred to give people a choice in case they really know what they are doing and that the edited archive will not work the 99.9% of times.
Please don't force me to remove that useless option from quickbms :)

Long story short: if you need to use the "force" option it means you are doing something completely wrong and you will receive no support.

That's the clear position about quickbms, but in your case, as already explained, you don't need quickbms to do something simple like that :)
Use a hex editor and do the job in one minute.


Top
   
PostPosted: Mon Oct 08, 2018 9:20 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
I would like to introduce a better way to use the reimport and reimport2 modes by performing some additional checks.

1)
If you use reimport.bat for reimporting 1000 files and one of them is bigger than the original then the reimporting will be interrupted.
Possible solution:
- ask the user if he wants to retry (y/N/r/force) so he can edit the file and retry without running the process again

2)
If you have extracted 100000 files and have edited only one of them, quickbms will try to reimport all the 100000 files... it may be a problem for modders because deleting the non-edited files before reimporting may be a difficult (boring) process in some games.
Possible solution:
- calculate an hash (sha1 or sha256) on the archived file and an hash of the one on disk, compare them and reimport the file only if it's different. obviously if the size is different there is no need to make the hash.
While this solution seems nice, it has the disadvantage that basically quickbms needs to read (and decompress, just a full extraction without disk writing) the input file and compute the hash spending time and resources, multiply it by two since the same is done on the file on disk too.
sha1 is faster than sha256 and in this non-security context it's probably the best choice since no collisions are expected while editing a file.
I may implement this feature as an option for quickbms but I don't know if I have to add it to reimport.bat and reimport2.bat


Top
   
PostPosted: Tue Oct 09, 2018 3:31 am 
User avatar

Joined: Wed Jul 19, 2017 5:04 am
Posts: 136
Shokoniraya wrote:
michalss wrote:
I dont think you understand the reimport2 method. There is no such a force needed neither necessary as aluigi told you, when file is bigger its automatically go to end of the archive and in data table rewrite is size and offset, etc or other props needed. This method can't be used in same cases of segmented archives i guess..


dear michalss: reimport2 means; set new offset and size and i totaly know that
and force mode means replace raw file

but we cant do it on single files white reimport2!
need a command option to replace raw files like force mode, not write, just like text files! and just adding a tag for do this thing...

go to end of the archive and in data table rewrite is size and offset: and set old data as "00 00 00 00", i know that.


yeah, right. but you can't do it with QuickBMS
Try to learn a language for making your own Tools

QuickBMS can't help in such thing: Text Files, Single Files, converting an File to another File Like PNG to tex. replace raw_data

_________________
Be Good, Even in distressing or defamation and chill, Even Everyone's Called You a Bad Guy Or Lier and Mad


Top
   
PostPosted: Tue Oct 30, 2018 7:07 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
quickbms 0.9.1 will use the Capstone library as disassembler and will support every architecture supported by the library, so not just x86.


Top
   
PostPosted: Wed Oct 31, 2018 4:27 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
reimport2 will work also with archives having the TOC at the end, maybe not 100% of times but most of them :)


Top
   
PostPosted: Thu Nov 01, 2018 9:31 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
I still have no release date for quickbms 0.9.1 but at least I have finished to write the (temporary) changelog (the list of things fixed and added).
Even today I have fixed some bugs that have been casually found during a test.
I have also updated some libraries.

The only priority at the moment is beta testing for spotting new issues.
Hopefully next week I will have good news :)


Top
   
PostPosted: Fri Nov 02, 2018 6:12 am 

Joined: Sun Aug 10, 2014 12:49 pm
Posts: 271
aluigi wrote:
I still have no release date for quickbms 0.9.1 but at least I have finished to write the (temporary) changelog (the list of things fixed and added).
Even today I have fixed some bugs that have been casually found during a test.
I have also updated some libraries.

The only priority at the moment is beta testing for spotting new issues.
Hopefully next week I will have good news :)



Are you also planning release fixed DDL with compress suport pls ? Waiting for this . :) thx


Top
   
PostPosted: Fri Nov 02, 2018 8:47 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9242
Sure


Top
   
PostPosted: Sun Nov 04, 2018 6:37 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 55
Removed


Last edited by Shokoniraya on Thu Nov 15, 2018 5:36 am, edited 1 time in total.

Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 374 posts ]  Go to page Previous 115 16 17 18 19 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
cron
Powered by phpBB® Forum Software © phpBB Limited