ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Wed Oct 21, 2020 7:02 am

All times are UTC




Post new topic  Reply to topic  [ 604 posts ]  Go to page Previous 127 28 29 30 31 Next
Author Message
PostPosted: Sat Sep 26, 2020 7:33 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
That thing of the memory files you suggested is complicated and without any real advantage.

URI/URL must use / and not \, that's part of the HTTP protocol.


Top
   
PostPosted: Sat Sep 26, 2020 8:52 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 325
anyway, it's really helpful to have such option in QuickBMS
(sorry, i wanted to change my previous comment but i write a clear one)
without any real advantage? names are more explicit than memory numbers
Code:
set MEMORY_FILE1 string "1"
set MEMORY_FILE2 string "2"

math COUNT = 2
string MEMORY_SLOT p "MEMORY_FILE%s" COUNT
print "%MEMORY_SLOT%"
log memory_test1 0 1 MEMORY_SLOT #MEMORY_SLOT is MEMORY_FILE2, but it just acting as 0 file number, like this:(log memory_test1 0 1 0)

log memory_test2 0 1 MEMORY_FILE%COUNT% #not working, result must be 2, but output is 1, %COUNT% not working as MEMORY_FILE file number at all



and about names for MEMORY_FILE
result should be "take", but it does not using names for MEMORY_FILE
that means it's acting as unnumbered MEMORY_FILE (it's just a suggestion, using names for MEMORY_FILE could be useful)
Code:
set MEMORY_FILE_take string "take"
set MEMORY_FILE_last string "last"

log memory_name 0 4 MEMORY_FILE_take #result should be "take"


Top
   
PostPosted: Wed Sep 30, 2020 2:42 am 

Joined: Tue Sep 08, 2020 3:31 pm
Posts: 11
Thought about File Ripper

I catched up a few lines about to add such feature in QuickBMS. It's time expansive for sure and the solution would be a community-based
task. Unfortunately I'm not blessed with coding skills so I offer support in alternative ways. The wheel was already invented so I think it
would be less time consuming if you could implement a existing open source based file ripper and customize it instead making a neat one.


Top
   
PostPosted: Wed Sep 30, 2020 9:34 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
@Shokoniraya
It would be difficult to implement that thing because all the files are just numbers in the bms language.
And the benefits are basically none.
You can just add a comment after each MEMORY_FILE to know what's meant for.

@ExtractResponseUnit
That would be nice if it's a separate tool than quickbms because it needs a different logic.
File rippers are difficult because sometimes there are no ways to calculate the size of the file, and scanning the magic values after the found one would produce many false positive.
I stay away from writing rippers :)

Instead, regarding the binary String feature implemented in the beta, I just noticed that it's not compatible with those situation where we have something like:
Code:
getdstring PATH 0x20
getdstring NAME 0x20
string PATH + NAME

The result is just the original PATH and not PATHNAME because in binary mode the whole variables are concatenated, so the result is 0x20 bytes of PATH followed by 0x20 bytes of NAME.
Not sure what solution I'm going to implement but probably a parameter specified by the user for setting the binary mode.
Currently String has '0' as special prefix for the operator, what about using it also for the binary mode?
Like: string PATH 0+ NAME
Obviously the mode using NUL-delimited strings will remain the default as it is now.


Top
   
PostPosted: Wed Sep 30, 2020 10:24 pm 

Joined: Tue Sep 08, 2020 3:31 pm
Posts: 11
aluigi wrote:
@ExtractResponseUnit
That would be nice if it's a separate tool than quickbms because it needs a different logic.
File rippers are difficult because sometimes there are no ways to calculate the size of the file, and scanning the magic values after the found one would produce many false positive.
I stay away from writing rippers :)


I don't know how to compile pascal source code but I think someone on this forum does so here we go . . .


Attachments:
HyperRipper (Source).rar [222.13 KiB]
Downloaded 23 times
Top
   
PostPosted: Fri Oct 02, 2020 1:29 am 

Joined: Thu Aug 07, 2014 10:28 pm
Posts: 310
Does quickbms support these hash functions?
they are used in unity.
https://github.com/facebook/folly/tree/ ... folly/hash

referenced here
https://github.com/Unity-Technologies/U ... okyHash.cs


Top
   
PostPosted: Fri Oct 02, 2020 3:10 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
Ok I just added spookyhash, murmurhash and xxhash in Encryption (not available in the online build yet).
Some of them are also available in the crc engine


Top
   
PostPosted: Sat Oct 03, 2020 12:31 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 325
sir aluigi, i found out that current beta version of QuickBMS has a problem

Code:
log MEMORY_FILE6 0 0
#or even set MEMORY_FILE6 binary ""
put MY_VAR long MEMORY_FILE6


but it says: Error: MEMORY_FILE6 has not been used/declared yet
can't make a empty MEMORY_FILE?
there is no problem in 0.10.1 version

and one other problem, i've attached a bms file in this comment, QuickBMS can't download this file, i can download it from any downloader (it's a direct link), can you check it?
thank you


Attachments:
link_bms.zip [328 Bytes]
Downloaded 29 times
Top
   
PostPosted: Sun Oct 04, 2020 12:08 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
Good catch on that MEMORY_FILE problem, I'm checking it right now.

I tried passing download/file.php?id=9258 to quickbms 0.11 as input file and it correctly downloads it, are you talking to some different problem?


Top
   
PostPosted: Sun Oct 04, 2020 12:16 am 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 325
yes, i'm takling about some direct links that can't download it with QuickBMS

please download attached file in my previous comment (link_bms.zip) and open zip file, then use test_link.bms
it's a bms file to download a direct link, but can't download it with QuickBMS by using that script

i'm talking about test_link.bms file (bms open "." ""), not attached zip link


Top
   
PostPosted: Sun Oct 04, 2020 5:57 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
Just checked and the behavrious of quickbms is correct.
In short your link is a huge file of 2147483648 bytes, so quickbms tries to allocate such amount in memory for downloading it and fails.

The maximum amount you can allocate is 2147221471 bytes (Content-Length field) which is crazy because quickbms is NOT a downloader.
It will not tell you if the file exists but it will first download it completely.


Top
   
PostPosted: Sun Oct 04, 2020 5:55 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
I just updated the beta on the website.
I did some changes to the multidimensional arrays (VAR[i] and VAR[i][j]) so please check them too.


Top
   
PostPosted: Sun Oct 04, 2020 11:25 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 325
about huge files, what about 4gb version? has same problem since it should handle unit

i tested array in some different ways, looks like works fine, can you tell some detail about those changes?


Top
   
PostPosted: Mon Oct 05, 2020 5:20 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
Not sure if 4gb bypasses it but for sure you should not use quickbms for downloading 2Gb files :)

Regarding the changes, they are mainly programming stuff without additional feature.

One change was that now VAR[] is considered a stand-alone variable itself while before it was a multidimensional array with a "" index that basically is just the same as now.
My plan is that maybe I can use it as a sort of way to receive the total number of entries or just leave it as-is.

I also moved a function that allocates memory for the arrays (add_multi_dimensional) because it was leaking lot of RAM for nothing.
Basically just this but I'm sure it may be possible to optimize it even more.


Top
   
PostPosted: Mon Oct 05, 2020 11:33 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 325
i think you should leave it be same as is (sorry for my limit grammar)
and just a question, can you check if it's possible to optimize read/write function, making it faster?

and one more thing, some of files are read-only, i know it can simply disabled, but can you do something about Error: Permission denied? (passing read-only, i mean if input file was read-only, then quickbms disable read-only for that file)


Top
   
PostPosted: Tue Oct 06, 2020 6:34 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
What you mean with making faster read/write?
Do you have an example?

Good idea the "read-only to write" thing, going to add it soon.


Top
   
PostPosted: Tue Oct 06, 2020 7:13 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
Ah that permission denied is NOT related to read-only files.
It's related to the permissions of the folder, long story short I will never touch that.
But I did the read-only thing when using -w, in that case quickbms showed the message: Do you want to create it from scratch (y/N)?


Top
   
PostPosted: Tue Oct 06, 2020 11:01 pm 
User avatar

Joined: Sat Sep 15, 2018 5:22 am
Posts: 325
sir aluigi, i found a problem in savepos, i know it can be done with a little more coding in bms file. but can you check it? i think savepos has problem in asize (reimport mode)

i've attached a zip file in this comment
there is a savepos.bms in zip file, put quickbms.exe in savepos_problem folder, and goto file_content folder and open a file with name sec_3 in hex editor, and add a few byte (or remove a few byte) from that file
and click on _import.bat, and you will see that END_OFFSET (in bms file) not changed in reimporting (asize problem in savepos)

also, there is same problem with xmath and math too, becuase they will not change in reimporting

can you check other commands that related to such cases too?


Attachments:
savepos_problem.zip [2.53 KiB]
Downloaded 21 times
Top
   
PostPosted: Wed Oct 07, 2020 12:57 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
There was for sure a problem with asize because I fixed it few hours ago while checking another topic in which clog added a 32bit field into the compressed data (but I don't think it's the problem you refer).

Regarding reimport3 and savepos... it's a mess.
quickbms already resets the variables containing the asize value without considering any math operation on them.
For savepos it would be complicated to adjust them.


Top
   
PostPosted: Fri Oct 09, 2020 11:25 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11680
New beta is available:
http://aluigi.org/beta/quickbms_beta.zip

(the beta is complete: full source code and both the exe compiled for release)


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 604 posts ]  Go to page Previous 127 28 29 30 31 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited