ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sat Oct 21, 2017 1:18 am

All times are UTC




Post new topic  Reply to topic  [ 251 posts ]  Go to page Previous 19 10 11 12 13 Next
Author Message
PostPosted: Sat Apr 29, 2017 6:05 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
Good catch, it seems just a bug in the new "tcc" feature (the runtime C compiler) of quickbms.
The game has tons of files and there is a certain amount of memory consumed and not freed at each cycle so you get the memory allocation error when the process has already allocated some Gb of memory.
I was ready to suggest to use -9 (that limits these types of memory leaks) but quickbms even crashes with it so it looks like a lame bug that I have to solve for the next version.

Unfortunately there are no temporary work-arounds to complete the extraction.


Top
   
PostPosted: Sat Apr 29, 2017 6:10 pm 

Joined: Sat Apr 29, 2017 4:24 pm
Posts: 2
No sweat, I don't mind waiting for next version. Thanks for your time on this issue. It's been a huge help!


Top
   
PostPosted: Tue May 09, 2017 6:51 am 
User avatar

Joined: Fri Aug 08, 2014 1:06 am
Posts: 333
i think i found a bug with the findloc command, my last entry is ignored :o
i will attach a sample and the script :)
*solved and removed*
i didn't see anything in quickbms.txt about there being a limit to how many
times you can use it or where you can/can't use it. :?

_________________
Quickbms.com link thread
Search bms scripts


Last edited by Acewell on Tue May 09, 2017 7:47 am, edited 1 time in total.

Top
   
PostPosted: Tue May 09, 2017 7:03 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
It's a conflict between the searched string and the name of the first variable.
Fix: replace the name of the first variable (RYHP) with NEXT_RYHP

Yeah I know it's boring but one of the rules of quickbms is that name of variables and constant strings (even the binary ones) are handled in the same way.
Even I forgot that and it required some tests before fixing it :D


Top
   
PostPosted: Tue May 09, 2017 7:42 am 
User avatar

Joined: Fri Aug 08, 2014 1:06 am
Posts: 333
okay so in newby terms, avoid making the variable and search strings the same, even if one is binary and the other is a string?
thanks :D

_________________
Quickbms.com link thread
Search bms scripts


Top
   
PostPosted: Tue May 09, 2017 8:10 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
Exactly, variable names and constant strings (like string or "string" or "\x73tring") MUST be ever different.


Top
   
PostPosted: Sun May 28, 2017 3:26 pm 

Joined: Sun Oct 09, 2016 6:27 pm
Posts: 23
I think I found a bug with the -P command or the Rpa Renpy Nopython script:

When extracting an RPA file that contains Cyrillic characters in some of the file names they did not extract correctly so tried adding -P UTF8 (unless I'm wrong RPA files are always UTF-8 encoded) but the result was still wrong, next I tried every other UTF and Cyrillic option from this table using both Identifier and .NET Name values, I also tried adding the codepage "utf8" and codepage 1251 string in the BMS script with the same result.

For example the correct file name is: Armature-трет руки_0 as can be seen in the attached file table, but the files are extracted as: Armature-трет СЂСѓРєРё_0, Armature-трет руки_0 or Armature-трет руки_0.


Attachments:
TEMPORARY_FILE.7z [45.69 KiB]
Downloaded 19 times
Top
   
PostPosted: Sun May 28, 2017 5:02 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
Yeah that's something that require to be checked for the next release because it appears to be a bug (utf8 is correct but some chars are clearly wrong).
Added in my TODO.


Top
   
PostPosted: Tue Jun 27, 2017 2:38 am 

Joined: Tue Jun 27, 2017 2:35 am
Posts: 1
Tried running on OS X El Capitan 10.11.6, and all I get is this text in the Terminal:
Code:
/Users/[redacted]/Downloads/quickbms_0.8.0_macosx/quickbms ; exit;
[computername]~ [redacted]$ /Users/[redacted]/Downloads/quickbms_0.8.0_macosx/quickbms ; exit;
dyld: Symbol not found: __ZdlPvm
  Referenced from: /Users/[redacted]/Downloads/quickbms_0.8.0_macosx/quickbms (which was built for Mac OS X 10.11)
  Expected in: /usr/lib/libstdc++.6.dylib
 in /Users/[redacted]/Downloads/quickbms_0.8.0_macosx/quickbms
Trace/BPT trap: 5
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.



Top
   
PostPosted: Tue Jun 27, 2017 1:11 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
Another user reported an unresolved symbol in the exe I uploaded (if that's the exe you are using), but I can't find the post so probably it was by email.
I guess you have to compile it by yourself


Top
   
PostPosted: Sun Jul 02, 2017 11:38 am 

Joined: Thu Aug 07, 2014 10:28 pm
Posts: 131
What am i doing wrong to get the encryption key correctly.
If i print the key it is changing but all the log commands I do get the same output.
Code:
putarray 0 0  "\x98\x3b\xa6\xcc\xa7\x52\x32\xf9\x23\xbb\xe8\x7d\x39\x6c\xb7\x4e"
putarray 0 1  "\x13\xc7\xe8\x81\xd9\x8c\x75\x16\xf3\x2f\xbb\xf4\x21\x56\xb1\xa6"
putarray 0 2  "\x38\x50\x7b\x33\xee\xe5\xf0\x53\x4c\x5d\x2f\xc7\xf1\x65\xb8\x4a"
putarray 0 3  "\xc4\x55\x7f\x7c\xba\xb4\x42\x91\xee\x51\x2e\x37\xeb\x23\xaf\x54"
putarray 0 4  "\x23\xeb\x2d\xe6\xf7\x49\x92\x7c\x82\xfd\xec\x8e\xdc\x9f\x3e\xbc"
putarray 0 5  "\xcb\x49\xd8\x7c\xcc\x59\x7b\xca\x2d\xe2\x24\xb7\x19\x36\x37\x4d"
putarray 0 6  "\x2d\x3a\xfb\x28\x54\x21\x50\xb4\xa0\x77\xb8\xbb\xbf\xe8\xb1\x67"
putarray 0 7  "\xe3\x6f\xb2\xef\x65\x2b\xa2\x3a\x58\x2a\xba\x1f\xae\x68\xec\xbc"
putarray 0 8  "\xe7\x1e\x3d\xd6\xf5\xe2\x87\x9f\x68\xa1\x8b\xbb\xc7\xd4\xf2\x7d"
putarray 0 9  "\x3e\x3a\x10\xe4\xc1\x7b\xdf\x72\x39\x46\x40\x16\xfe\x94\x6b\xb5"
putarray 0 10 "\x20\x59\x25\xb9\xa6\x6b\x77\xc0\xbd\x4e\xe0\xbd\x1a\x25\x64\x3b"
putarray 0 11 "\x3f\xd0\x85\x35\x18\xd3\x8c\x59\x89\xc6\xd6\x61\x82\xb8\x5f\x75"
putarray 0 12 "\x25\x2b\xbc\xe5\x3a\xb3\x8b\x75\xb1\x2c\x7a\xdf\x98\xe6\x57\x8b"
putarray 0 13 "\x81\x6a\xcb\xb0\x8b\x17\x15\x10\x40\x53\x7b\x4a\x8a\xce\x77\x18"
putarray 0 14 "\x44\xc9\xcb\xd2\xb1\x76\xed\x2b\x18\xc6\x95\xc9\xd9\xda\x5a\xec"
putarray 0 15 "\x6b\x88\xa6\xd4\x16\x57\x3d\xd0\xb2\x4d\x1f\xf2\x48\x73\x17\x44"

get FILES long MEMORY_FILE
for i = 0 < FILES
    get NAMESZ long MEMORY_FILE
    getdstring NAME NAMESZ MEMORY_FILE
    get SIZE threebyte MEMORY_FILE
    get FLAG byte MEMORY_FILE
    get DUMMY long MEMORY_FILE  # 0xffffffff or zero
    get OFFSET longlong MEMORY_FILE
    math OFFSET += BASE_OFF
    #print "%NAME% %FLAG%"
    set EXT extension NAME
    if EXT == "png"
    print "%NAME% %FLAG%"
    callfunction LAME_GUESS_IDX2 1
    endif
    encryption "" ""
next i

startfunction LAME_GUESS_IDX2
    for KEY_IDX = 0 < 16
        getarray KEY 0 KEY_IDX
        string KEY x KEY
        encryption blowfish KEY "" 0 0x10
        set BNAME NAME
        string BNAME + KEY_IDX
        log BNAME OFFSET 4
    next KEY_IDX
    break
endfunction


Top
   
PostPosted: Thu Jul 06, 2017 3:16 am 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 965
Okay, now this is where I can't grasp your programming script.

So, I'm writing a script to split multiple files out of a .sgb/.sgh file into individual .sgb/.sgh pairs. However, because setting up the header was a bit complex than I thought, I thought of doing this:
Code:
   savepos NAME_OFFPOS1 MEMORY_FILE5
   xmath NAME_OFFPOS_F1 "NAME_OFFPOS1 + 0x10"
   goto NAME_OFFPOS_F1 MEMORY_FILE5
   savepos NAME_OFFPOS_OFF1 MEMORY_FILE5
   putct NAME_NAME1 string -1 MEMORY_FILE5
   put 0 byte MEMORY_FILE5
   savepos NAME_OFFPOS_OFF2 MEMORY_FILE5
   putct NAME_NAME1 string -1 MEMORY_FILE5
   goto NAME_OFFPOS1 MEMORY_FILE5
   put 0 short MEMORY_FILE5
   put 0x3000 short MEMORY_FILE5
   put NAME_OFFPOS_OFF1 long MEMORY_FILE5
   put 0 short MEMORY_FILE5
   put 0 short MEMORY_FILE5
   put NAME_OFFPOS_OFF2 long MEMORY_FILE5
However, this is what actually happens:
Code:
. 00000000000000b8 putct   NAME_NAME1 "MS_01_2" -1
. 00000000000000bf put     0          0x0000000000000000 1
. 00000000000000c0 putct   NAME_NAME1 "MS_01_2" -1
. 00000000000000b8 put     0          0x0000000000000000 2
. 00000000000000ba put     0x3000     0x0000000000003000 2
. 00000000000000bc put     NAME_OFFPOS_OFF1 0x00000000000000b8 4
. 00000000000000c0 put     0          0x0000000000000000 2
. 00000000000000c2 put     0          0x0000000000000000 2
. 00000000000000c4 put     NAME_OFFPOS_OFF2 0x00000000000000c0 4
So you see, the script writes these names through a MEMORY_FILE5 file, but not through the NAME_OFFPOS_F1 offset as expected. Instead, later variables are then written in place of two "putct" lines and one 8-bit(one byte) "put" value on the same MEMORY_FILE5 file.

To elaborate, I want to write these names on an offset that goes after the offset in which a blank space of 16 bytes are reverse for these six "put"(two 16-bit values and one 32-bit value) variables - all into MEMORY_FILE5.

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
PostPosted: Thu Jul 06, 2017 11:37 am 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 965
Turns out it's just me. I started working around this by removing these lines:
Code:
   xmath NAME_OFFPOS_F1 "NAME_OFFPOS1 + 0x10"
   goto NAME_OFFPOS_F1 MEMORY_FILE2
and replacing it with these lines:
Code:
   put 0 long MEMORY_FILE2
   put 0 long MEMORY_FILE2
   put 0 long MEMORY_FILE2
   put 0 long MEMORY_FILE2
Still I won't edit the above post into nothing though, my stupidity will remain there for "historical reasons".

---

Okay, here I am again with my problems with quickBMS.

So I'm pretty close of completing my ".sgh/.sgb/.sgd splitter" script, but the thing is, the archive I want to split is about 512+ files. With MEMORY_FILE stuff(although not with the "get"-like commands thankfully), the script can only generate 256 of these .sgh/.sgb files and it cannot go beyond that number. Here is the proof:
Code:
Error: incomplete input file 1: G:\SONY\PS3\GENJI2\PS3_GAME\USRDIR\DT64\SOUND\G2
DEMO.SGB
       Can't read 111568896 bytes from offset 0000000025602ce0.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1    42%   627059936  1477753216 . offset 0000000025602ce0

Last script line before the error or that produced the error:
  289 log MEMORY_FILE WAVE_OFFSET1 WAVE_SIZE2 1
So, despite the "unreachable offset" this error tells me I can "reach" that offset just fine through an hex editor, which means that I'm left with a dead-end over an archive whose data is entirely intact.

So to summarize, only 512 files(514 in G2DEMO.SGB/G2DEMO.SGH this case) are allowed to be written while on MEMORY_FILE mode and any number beyond that is overkill for quickBMS at this point. Is there any way around this?

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
PostPosted: Wed Jul 12, 2017 2:49 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 228
Hi, open command in new version of quickbms doesn't work well
Code:
Open FOLDER NAME [FILENUM] [EXISTS]

When i use code like this
Code:
open "." NAME 2 CHECK
 if CHECK == 0
    some actions
else
    some actions
endif

i get messages that ask to press Y/N for each file (https://puu.sh/wHJv5/2b271a821e.png). Older versions of quickbms didn't have this "bug" and never asked any confirmation. Please fix it, cause it makes [EXISTS] argument useless.


Top
   
PostPosted: Fri Jul 14, 2017 5:40 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 228
Findloc has a searching bug for some offsets - https://puu.sh/wJwIr/bfc1ff14e3.png

I've tried to search that string apart with script
Code:
string ID p= "%08x" 0xd0006842
string ID += "="
#goto 1
FindLoc STR_OFF string "d0006842="

and this is what i've got - https://puu.sh/wJwYg/3952738b94.png
As you can see when i search from beginning of file bms can't find anything, but if i change position to 1 (goto 1) it works fine.

P.S.
0.7.7 and 0.8.0 gives the same result.


Top
   
PostPosted: Thu Jul 20, 2017 4:07 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
The EXISTS field of the Open command gave me no problem till now so I suspect there is something else, maybe are you using the reimport feature or the -w write mode?
The "do you want to create it from scratch" is used just when you must write files and not during the normal reading operations:
Code:
    if(g_write_mode) {
        filez->fd = xfopen(fname, "r+b");    // do NOT modify, it must be both read/write
        if(!filez->fd) {
            if(g_reimport) {
                if(error) STD_ERR(QUICKBMS_ERROR_FILE_WRITE);
                return -1;
            } else {
                fprintf(stderr, "\n"
                    "- the file %s doesn't exist.\n"
                    "  Do you want to create it from scratch (y/N)?\n"


Regarding the findloc problem you are right, if the string starts at offset 0x9fdc it's not seen, while it works if it's in other positions... good catch!


Top
   
PostPosted: Thu Jul 20, 2017 4:14 pm 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 965
aluigi wrote:
The "do you want to create it from scratch" is used just when you must write files and not during the normal reading operations:
Code:
    if(g_write_mode) {
        filez->fd = xfopen(fname, "r+b");    // do NOT modify, it must be both read/write
        if(!filez->fd) {
            if(g_reimport) {
                if(error) STD_ERR(QUICKBMS_ERROR_FILE_WRITE);
                return -1;
            } else {
                fprintf(stderr, "\n"
                    "- the file %s doesn't exist.\n"
                    "  Do you want to create it from scratch (y/N)?\n"
I've never seen that before. Is this the solution to my problem or are you going with this "feature" on the next version?
(Pardon my ignorance.)

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
PostPosted: Thu Jul 20, 2017 4:19 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
That's for spider91.

I gave a quick look at your post but... headache :)


Top
   
PostPosted: Thu Jul 20, 2017 7:00 pm 

Joined: Sun Aug 24, 2014 5:26 pm
Posts: 228
aluigi wrote:
The EXISTS field of the Open command gave me no problem till now so I suspect there is something else, maybe are you using the reimport feature or the -w write mode?
The "do you want to create it from scratch" is used just when you must write files and not during the normal reading operations:
Code:
    if(g_write_mode) {
        filez->fd = xfopen(fname, "r+b");    // do NOT modify, it must be both read/write
        if(!filez->fd) {
            if(g_reimport) {
                if(error) STD_ERR(QUICKBMS_ERROR_FILE_WRITE);
                return -1;
            } else {
                fprintf(stderr, "\n"
                    "- the file %s doesn't exist.\n"
                    "  Do you want to create it from scratch (y/N)?\n"


Regarding the findloc problem you are right, if the string starts at offset 0x9fdc it's not seen, while it works if it's in other positions... good catch!


Not reimport, but yes, i use -w mode. Anyway i don't think that's a good solution even with write mode, cause we have [EXISTS] argument specially to check it.
Or maybe it would be better this message to appear only when [EXISTS] argument is not used?


Top
   
PostPosted: Thu Jul 20, 2017 7:57 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6885
That code probably doesn't consider the EXISTS field (variable "error" in the fdnum_open function) in a correct way or, more probably, it was meant for a different usage that I don't remember.
Anyway yes I can easily fix it by adding an "if(!error) return -1;" just before fprintf.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 251 posts ]  Go to page Previous 19 10 11 12 13 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited