ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Wed Oct 28, 2020 8:20 pm

All times are UTC




Post new topic  Reply to topic  [ 455 posts ]  Go to page Previous 119 20 21 22 23
Author Message
PostPosted: Fri Aug 21, 2020 9:38 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11696
quickbms already uses the latest versions of the compression libraries.
It also already uses the best compression levels, example for lzo1x:
lzo1x_999_compress(in, insz, out, &len, wrkmem);

Using a better compression level doesn't mean that the output will be ever smaller, sometimes it can be bigger with some samples.

zlib is slow for various reasons.
Differently than other packing tools, QuickBMS is a reimporter and so its priority is the final size of the data which means using maximum compression for fitting the original size.
It even uses 3 different types of zlib if the final data is bigger than the original which means a huge amount of time.

I guess that you have that problem only with reimport.bat since reimport2.bat should just use the normal compression.

Searching binary data with quickbms is difficult because the tool is not meant for that.
I can try to improve findloc binary but I doubt in any better result.

What is that "getbinary"???
getdstring already does the job


Top
   
PostPosted: Tue Aug 25, 2020 10:37 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
getdstring can't do that job

here a clear example
Code:
set MEMORY_FILE binary "\x00\x01\x00\x02\x00\x03"
get MEM_SIZE asize MEMORY_FILE

string HEX_STRING = ""
for i = 0 < MEM_SIZE
get BYTE_NUM byte MEMORY_FILE #same job as getvarchr to get every VAR in for-next
string HEX_BYTE p "%02x" BYTE_NUM #so it will make a real hex-string
string HEX_STRING + HEX_BYTE
next i
print "hex string: %HEX_STRING%"
string HEX_STRING h HEX_STRING #and it will make it byte, but its string! so \x00 will be a problem!

set MEMORY_FILE5 binary "\x00\x01\x00\x02\x00\x03"
findloc RESULT binary HEX_STRING MEMORY_FILE5 ""
print "Result: %RESULT%"


and can you do something to have some options to choose temp files for exe executing? some codec tools don't have [output] (example: program.exe -c [input] [output]), so output will have a random name


Top
   
PostPosted: Tue Aug 25, 2020 11:18 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11696
That's exactly the job of getdstring.
What you are showing is something different related to strings (strings are NUL delimited) and findloc which is still related to strings.
I can do something for Findloc and maybe a work-around for String when used with some operators.

This is a limitation that affects me too and I want to find a solution, but it's NOT related to getdstring :)

Regarding the other request I can't do anything because quickbms gets the result from the output file generated by the program.
quickbms doesn't act as hooker or hypervisor of the program, it just creates the input file, pass it to the program and read the generated output file.


Top
   
PostPosted: Tue Aug 25, 2020 5:20 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
and about temp names, temp files created in quickbms temp folder, if there was a way to have last temp file adress that created in folder, then problem will solve


Top
   
PostPosted: Wed Aug 26, 2020 1:02 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
aluigi wrote:
That's exactly the job of getdstring.
What you are showing is something different related to strings (strings are NUL delimited) and findloc which is still related to strings.
I can do something for Findloc and maybe a work-around for String when used with some operators.

This is a limitation that affects me too and I want to find a solution, but it's NOT related to getdstring :)

Regarding the other request I can't do anything because quickbms gets the result from the output file generated by the program.
quickbms doesn't act as hooker or hypervisor of the program, it just creates the input file, pass it to the program and read the generated output file.


and about getdstirng, that's right, that little example wasn't related to getdstring, but i'm saying that string will skip in \x00 (null delimited)
so if we use getdstring TEXT 15 on \x68\x65\x72\x65\x00\x61\x6E\x00\x65\x78\x61\x6D\x70\x6C\x65, then it will stop on \x00, but we need whole string, that's why im saying that there is a getbinary needed


Top
   
PostPosted: Sun Aug 30, 2020 4:03 pm 

Joined: Mon Oct 22, 2018 12:24 pm
Posts: 8
Trying to extract PSARC files using the latest quickBMS and brink.bms versions from a new-ish PS4 game. I had to use the 4GB exe version, because the PSARC file I wanted to unpack is over 7GB.

Code:
- error in src\extra\xalloc.c line 618: xdbg_malloc()
Error: memory allocation problem
       Not enough memory resources are available to process this command.


It can extract some psarc files pretty easily. I guess the problem here is that the file it wants to unpack is way too big. Any workaround? Or do I just wait for a fix? Is there a way to continue unpacking after that point?


Top
   
PostPosted: Tue Sep 01, 2020 4:29 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
there is two problem that i have found in QuickBMS

1) bms_folder
BMS_FOLDER can't get script folder as well
my_script.bms is located in "G:\my_scripts\works\pak_format\my_script.bms"

and this is my command
"quickbms.exe" -o "G:\my_scripts\works\pak_format\my_script.bms" "my_file.bin" "output_folder"
result is "G:\my_scripts\works\pak_format" and it will work

but if i use it without a full name and drive and file name, result is "G:\my_scripts\works" for this command
"quickbms.exe" -o "my_script.bms" "my_file.bin" "output_folder"

i think QuickBMS removing last sub-folder or sub-name from bms adress
problem is that i can't use it if i write script name without full path


2) EXECUTE command
EXECUTE in comtype command has a problem, and maybe it can't works well with Temp folder on a windows user name that has space character in it (like my system user name)
because i get this error
@calc\calc.exe INPUT_NUMBER / 32 >C:\Users\************\AppData\Local\Temp\quickbms_00000da80000000181adcaa4289aa2b5.tmp
The system cannot find the path specified.


here a example: http://www.mediafire.com/file/ew70myc2t ... m.zip/file


Top
   
PostPosted: Wed Sep 02, 2020 3:03 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11696
Quote:
and about temp names, temp files created in quickbms temp folder, if there was a way to have last temp file adress that created in folder, then problem will solve

There are tons of processes creating files there :)

Quote:
Trying to extract PSARC files using the latest quickBMS and brink.bms versions from a new-ish PS4 game. I had to use the 4GB exe version, because the PSARC file I wanted to unpack is over 7GB.

That's off-topic here.

Quote:
so if we use getdstring TEXT 15 on \x68\x65\x72\x65\x00\x61\x6E\x00\x65\x78\x61\x6D\x70\x6C\x65, then it will stop on \x00, but we need whole string, that's why im saying that there is a getbinary needed

I'm working on the next version of quickbms just in these days and the results are interesting because the String command is now no longer limited to NUL strings.
Obviously there will be some incompatibilities with some scripts and some headaches in some situations, I will upload a beta quickbms.exe soon so you can try the new features.

Quote:
1) bms_folder
BMS_FOLDER can't get script folder as well

What's the current folder from where are you launching quickbms?
If I'm in g:\my_scripts\works\pak_format both the examples create output_folder where expected.

Quote:
2) EXECUTE command
EXECUTE in comtype command has a problem, and maybe it can't works well with Temp folder on a windows user name that has space character in it (like my system user name)
because i get this error

Good, this is definitely something to fix (and apparently related to the name(blah).exe bug)


Top
   
PostPosted: Thu Sep 03, 2020 1:52 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
about temp folder, yes, there was so many files on there, i'm not talking about last file that created in temp folder (based on time stamp), i mean last file that EXECUTE created it
quickbms dumping that file on temp folder, right? so it will be useful if get dumped file path from dumper function

and about bms_folder, i upload a example, please use it under two sub folder, like C:\folder_1\folder_2\bms_folder
http://www.mediafire.com/file/2zf956e2j ... r.zip/file

in just_script_name.bat, last sub folder is missing from result

and can you add LZX compressor too?

using -n in QuickBMS, is there any way to get file name in a download link that does'ny have any name?
becuase some links does not have any name or not a normal name like php file link
for examles, these links
"https://zenhax.com/download/file.php?avatar=2_1542023899.png"
"https://zenhax.com/download/file.php?id=9002"
"https://avatars1.githubusercontent.com/u/5502283?s=460&u=861e923455e5411cd3bfb07a8c533db777ea9276&v=4"

maybe they are using some java script method or redirected links
and i thing downloading a file from a server with QuickBMS is slow (downloading speed)


Top
   
PostPosted: Thu Sep 03, 2020 11:55 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11696
I still miss the purpose behind the name of the temporary files created by quickbms.
They are immediately deleted after the launched executable terminates and the content of the output file is read by quickbms.

I found and fixed the bug of BMS_FOLDER.

LZX is already available since years and in many variants.
Comtype:
- lzx, the version used in an old tool
- xmemdecompress, the most used in games
- lzxcab, used in the Microsoft CAB files (there is also a _delta version)
- lzxchm, used in the Microsoft CHM files (there is also a _delta version)
- cabextract, same as lzxcab I guess
- DS_LZX

The compressor is already available for xmemdecompress and ds_lzx.

Regarding the last question it's a bit complicated.
The library used by quickbms doesn't return the original name and even in that case I would have to choose between the original URL and the name of the file provided by the server.
Since quickbms is NOT a downloader and it only needs the content of the file, it's more appropriate to keep the original URL.
What I will do in the next version is improving the available names, example:
Code:
filename     file.php
fullname     https://zenhax.com/download/file.php?id=9002
basename     file
extension    php
fullbasename download/file
filepath     download


quickbms has the same download speed of anything else.
Probably you are comparing it with downloaders downloading the file with multiple connections which is totally unrelated.


Top
   
PostPosted: Sun Sep 06, 2020 6:08 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
sir aluigi, one other thing

can you update QuickBMS guess function for file extension? like mp3 files, or adding more guess for media files


Top
   
PostPosted: Sun Sep 06, 2020 9:03 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11696
MP3 files don't have an header since they are just a sequence of compressed frames without any "magic" value (often they start with byte 0xff but it's useless).
Some years after mp3 has been made it started to be used the ID3 struct for containing metadata like artist, name of the song and so on.
QuickBMS uses the ID3 "magic" for guessing the mp3 extension, so there is no extension if there is no ID3.

As far as I know quickbms already supports most of the format available, if something is not supported it's just because there is not "magic" to guess.

Let me know if you have suggestions about a specific format


Top
   
PostPosted: Sat Oct 17, 2020 3:03 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
found a problem
QuickBMS asks -w in slog exporting (tested with last beta)

here a script
viewtopic.php?f=12&t=14240&p=59519#p59519

i've attached a sample file too (in this comment)


Attachments:
localisation.zip [145.19 KiB]
Downloaded 10 times
Top
   
PostPosted: Thu Oct 22, 2020 4:52 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11696
Fixed ;)


Top
   
PostPosted: Sun Oct 25, 2020 3:59 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 335
problem solved (now, im sure UltraEdit broked my file, gotta leave it)


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 455 posts ]  Go to page Previous 119 20 21 22 23

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited