ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Thu Aug 16, 2018 8:54 am

All times are UTC




Post new topic  Reply to topic  [ 279 posts ]  Go to page Previous 18 9 10 11 1214 Next
Author Message
PostPosted: Wed Mar 08, 2017 12:59 pm 

Joined: Tue Dec 20, 2016 8:18 pm
Posts: 15
aluigi wrote:
"break" wasn't available in the original language.
So can't you just translate the reverse engineerd function into C code and use it (yeah "dumping" since there is no need of dll if it's simple) with calldll?

I can, but otherwise what for we have quickbms? For me personally quickbms is very handy because of its simplicity, it works out of the box and the most important - it doesn't take too much time to write script which will do the job.


Top
   
PostPosted: Wed Mar 08, 2017 1:12 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
I know.
Let's say the "break" problem gets fixed, it will remain the huge slowness issue affecting the bms code used for compression and encryption stuff (input->operations->output). Even 10 megabytes of input data can take minutes to be elaborated.
Regarding the readability, it's impossible to compare the length and readability of a function written in bms (long and chaotic) with the same written in C.

So the easiest and most elegant solution is just a precompiled function or dll used with CallDLL.
It's fast, you can embed it directly in the script and if you want to keep the source code you can just put it in a comment.
There is even the possibility to use it in the Log/CLog command through the Comtype and Encryption instructions, the input/input_size/output/output_size will be automatically handled.
The only negative point is that the user will be prompted to acknowledge the usage of the code, just that.


Top
   
PostPosted: Wed Mar 08, 2017 1:15 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
There are various examples on my website about calldll, the following covers both calldll and the commented original function:
http://aluigi.org/bms/tales_of_berseria.bms

While this is the most known example related to calldll used directly in log/clog:
http://aluigi.org/bms/rfactor2.bms


Top
   
PostPosted: Thu Mar 09, 2017 8:28 am 

Joined: Tue Dec 20, 2016 8:18 pm
Posts: 15
I still think that quickbms must have ability to do everything on its own, in other words it must become self-contained tool which doesn't require external bells and whistles.

And I really whish that in one day labels in conjunction with jump operator will be added as a new feature. Is it really that hard to add such feature?


Top
   
PostPosted: Thu Mar 09, 2017 12:34 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
Quote:
I still think that quickbms must have ability to do everything on its own, in other words it must become self-contained tool which doesn't require external bells and whistles.

In my opinion here there is a misunderstanding because quickbms is not an interpreter of a programming language, we have just a limited set of instructions with the final goal of extracting data from an input (file, network, memory and so on).
The problems raise when people start to use quickbms instead of a programming language.
I'm glad of that but making (for example) a 3d model converter in bms instead of python is not a good option.

For example this is a script I made years ago to implement the lzss algorithm in bms language: http://aluigi.altervista.org/bms/pure_l ... t_only.bms
It's huge and slow, much better to implement it as one-line pre-compiled binary to use with calldll.
Or better to use the native one, if people have custom lossless compression algorithms they want to see natively in quickbms then it's enough to contact me and I will add them immediately in the next release.

Quote:
And I really whish that in one day labels in conjunction with jump operator will be added as a new feature. Is it really that hard to add such feature?

My goal is to fix the "break" command, but I don't want to have something that will start to no longer work due to this change, and being "break" a sort of experimental instruction it's priority is really very low.

Currently the development of quickbms is focused on new algorithms to implement and bugfixes, so everything that doesn't require time and that will not affect the current features (fixing "break" is a bugfix with possible effects on the stability).
A feature like the one you proposed (which is basically the label+goto of C) is of no use to users, as you said that "may" be useful only if you reverse engineer a compression/encryption algorithm and want to implement it as bms language. But in that case the solution already exists: calldll.

Could you please tell me what's your problem with calldll?
It's easy and elegant, I really don't get what's the problem.


Top
   
PostPosted: Thu Mar 09, 2017 1:53 pm 

Joined: Tue Dec 20, 2016 8:18 pm
Posts: 15
Quote:
Could you please tell me what's your problem with calldll?
It's easy and elegant, I really don't get what's the problem.

The problem is that not always you can have access to all needed toolchain for that. And when you want to do something only once, is speed really important? Absolutely no. When job is done, the used script becomes pretty much useless and the only thing you can borrow further is your experience. But if I really need to use programming language, then I can use it from start to finish, right? Why do I need quickbms then? But no, I need it because it's much easier to deal with, no need to bother about dependencies, compiling and debugging. Convenience is the most important factor here, the less dependencies the better.


Top
   
PostPosted: Thu Mar 09, 2017 2:06 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
Can you provide some context?
I mean what you are doing, what programming language do you know, how big is the function you are reverse engineering, why quickbms would be better than python or C, and so on.

I'm thinking to what I can do, maybe there are easy solutions that I can implement in the next version.
If a 100% working "break" solves all the problems then I will put it at highest priority and finally closing this bug.
"Label" + "goto" is probably possible (I mean that it's probably not a huge work) if "break" works, obviously "goto" would not be used because already exists that instruction, maybe it can be an additional feature of "break" or "continue" (yeah there is even an experimental continue instruction), like "break label1" or "continue label2".
Just thinking...


Top
   
PostPosted: Thu Mar 09, 2017 2:43 pm 

Joined: Tue Dec 20, 2016 8:18 pm
Posts: 15
aluigi wrote:
Can you provide some context?
I mean what you are doing, what programming language do you know, how big is the function you are reverse engineering, why quickbms would be better than python or C, and so on.


My main hobby is modding, I never concentrate my doings on one language because of that. Different games have different gameplay core, some use their own script language, some use common ones, some use C, so when I deal with a new game, I need to use its language and because most of programming languages have lots of similarities it's not that hard to learn rules of the new one.

And why I really asked for labels is because it can help to make quickbms more versatile. Of all decompression algorithms supported to this day I had only two very lucky cases when quickbms detected used compression of the files and those cases were pretty laughable, one was packbits-RLE and the second was deflate, lol, I could figure it out by my own if I wasn't so lazy in that day... So I thought that It would be great to have some sort of backdoor in quickbms where it will be possible to create custom decompression functions by converting disassembled code into script. Actually RISC processors, for example, have very simplistic instruction set, so the only thing left for such possibility is labels and jump.


Top
   
PostPosted: Thu Mar 09, 2017 2:55 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
Ok, I'm doing some tests with "break" after a quick fix that includes also the dormant "continue" instruction.
Everything seems ok but obviously I need more tests cases because I'm sure there is something missing or wrong, it would be too good to have fixed this issue in less than 5 minutes :D

If you can produce some scripts that I can test, it would be great.


Top
   
PostPosted: Thu Mar 09, 2017 4:51 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
Situation at the moment:
Currently it looks like "break" and "continue" work correctly.
Labels work only if they are below the current goto ("continue"), so label loc_1, continue loc_1 will not work.
I have the possibility to embed a runtime C compiler in quickbms, I have already tried it and works perfectly :)


Top
   
PostPosted: Thu Mar 09, 2017 10:14 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
aluigi wrote:
The hanging of put unicode with the default codepage is for sure a bug... I released quickbms 0.7.7 just few days ago so who knows when I will check this thing.

Using codepage 1251 before the put command worked.
You have also to use "set str unicode str" after "get str unicode".

Anyway are you sure you really want to use quickbms for string related stuff like this?

I'm checking this bug of the endless loop with codepage 1200 but it's not a bug of quickbms.
Basically MultiByteToWideChar is unable to return a valid character with this codepage and so quickbms tries to give it an input which increases its size till the maximum length of the string, and the result is a super-slow function (not an endless loop).
In short there is nothing I can do for real because the characters can't be decoded.
I can only avoid the super-slow code.


Top
   
PostPosted: Thu Mar 09, 2017 10:31 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 8609
I don't remember who reported the following bug:
Code:
log MEMORY_FILE 0 0
set NAME string "XXXinputXXX"
strlen NAMELEN NAME
filexor 0x3a
putdstring NAME NAMELEN MEMORY_FILE
print "%NAME%"
where the content of the original variable gets modified (xored) after using putdstring.
Anyway I have decided to not fix it. filexor is a core component that works just in that way and this is a very rare event that never happens during normal use (filexor + put* + reusage of the variable).
This post is just to keep the problem logged.


Top