ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Tue Dec 11, 2018 6:23 am

All times are UTC




Post new topic  Reply to topic  [ 330 posts ]  Go to page Previous 113 14 15 16 17 Next
Author Message
PostPosted: Tue Oct 23, 2018 3:34 pm 

Joined: Fri May 27, 2016 2:28 pm
Posts: 53
Does this mean that also will fixed a allocation error, appearing when 'log' many files into Memory file with final size more then 1 gb and next try to 'log' this?


Top
   
PostPosted: Wed Oct 24, 2018 2:12 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
It depends by the error because it looks like you are trying to decompress (clog) a big file and therefore quickbms must allocate a double amount of memory.
Let's say you have already extracted various files and the current one has a compressed size of 900Mb and an uncompressed size of 1.1Gb, that means:
- current memory (let's say 500Mb) + 900Mb + 1.1Gb

You can try using the -9 option of quickbms but it shouldn't change much.
Anyway it's not a bug or similar.

For the log command (without encryption) quickbms doesn't allocate memory.


Top
   
PostPosted: Wed Oct 24, 2018 6:57 pm 

Joined: Fri May 27, 2016 2:28 pm
Posts: 53
It once was in my one packing script with log command, no clog. Was used 3 memory files: for files table, for files data, and for process loaded files in some order. At the last stage, table and data paste together and export. No one file was compressed, but yes, to memory file with files table encryption was assigned.


Top
   
PostPosted: Fri Oct 26, 2018 4:09 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
So the problem was related to 3 big memory files that weren't able to be allocated in memory.
Unfortunately the memory is very limited, let's say less than 3Gb (it's a 32bit application).


Top
   
PostPosted: Tue Oct 30, 2018 11:02 pm 

Joined: Sun Oct 09, 2016 6:27 pm
Posts: 28
aluigi wrote:
bugmenot wrote:
I think I found a bug with the -P command or the Rpa Renpy Nopython script:

When extracting an RPA file that contains Cyrillic characters in some of the file names they did not extract correctly so tried adding -P UTF8 (unless I'm wrong RPA files are always UTF-8 encoded) but the result was still wrong, next I tried every other UTF and Cyrillic option from this table using both Identifier and .NET Name values, I also tried adding the codepage "utf8" and codepage 1251 string in the BMS script with the same result.

For example the correct file name is: Armature-трет руки_0 as can be seen in the attached file table, but the files are extracted as: Armature-трет СЂСѓРєРё_0, Armature-трет руки_0 or Armature-трет руки_0.

This bug is the same reported in viewtopic.php?f=11&t=4797 and I confirm that it will be fixed in the upcoming 0.8.1 version that will be released this week :)
It seems the problem is still present in v9.0, any suggestions?

For example the HEX string 70 68 6F 6E 65 20 2D 20 D0 BA D0 BE D0 BF D0 B8 D1 8F 2E 70 6E 67 is saved as phone - РєРѕРїРёСЏ.png instead of phone - копия.png
Tested with GDPC and RGSSAD scripts on files containing Russian and Japanese file names.


Top
   
PostPosted: Wed Oct 31, 2018 7:51 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
It's an utf8 name which is correctly handled by quickbms, in fact my output name is perfect.

Script used for the test, and tested also with reading of name from file:
Code:
set NAME binary "\x70\x68\x6F\x6E\x65\x20\x2D\x20\xD0\xBA\xD0\xBE\xD0\xBF\xD0\xB8\xD1\x8F\x2E\x70\x6E\x67"
log NAME 0 0


You can get that wrong filename you specified only if you use -P 1251 which indeed is wrong because the correct charset here is utf8.


Top
   
PostPosted: Wed Oct 31, 2018 7:21 pm 

Joined: Sun Oct 09, 2016 6:27 pm
Posts: 28
In other words you are saying quickbms is running as non-Unicode program that's assuming it's running under 1250 encoding instead of checking the local system non-Unicode settings and taking them in account?

P.S. After testing the provided sample I get a file named "phone - копия.png" which probably means there is some other problem when reading the file name from the archive.

I've attached a small test GDPC sample file.


Attachments:
File comment: 7ziped GDPC sample file
asteroids.7z [21.83 KiB]
Downloaded 5 times


Last edited by bugmenot on Wed Oct 31, 2018 7:55 pm, edited 2 times in total.
Top
   
PostPosted: Wed Oct 31, 2018 7:35 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
Eh? No, quickbms uses utf8 by default, which is the codepage used for the string you provided.

Both the gdpc and rgssad have no hardcoded codepage in them and they don't even use any "unicode" conversion like "get NAME unicode" or "set NAME unicode NAME".
The scripts with hardcoded codepage (932) are: brave_soul.bms, ddworks.bms, hunex.bms and norn9.bms.

If you are experiencing that problem, please tell me how to replicate it step-by-step and upload the files you are using.

There is no problem reading filenames from archive, you can test it with a simple script like the following on a file containing the raw filename like the one you provided:
Code:
get SIZE asize
getdstring NAME SIZE
log NAME 0 SIZE


Top
   
PostPosted: Wed Oct 31, 2018 7:59 pm 

Joined: Sun Oct 09, 2016 6:27 pm
Posts: 28
Steps to reproduce:
Drag the GDPC.bms over quickbms.exe v9.0
select the asteroids.pck file from previous post
click save

Testing on a Win7 VM with non-Unicode language set to 1251. Switching to English (USA) produces a file named "копия.png".

P.S. The second sample is working fine as well.

P.P.S. It seems the problem is related to filenames that contain a directory in them. If I use a file containing "копия.png" the second example is working fine, but if it contains "res://копия.png" I get the wrong name.
"1/копия.png" produces the correct name but "1копия/копия.png" results in "1РєРѕРїРёСЏ\РєРѕРїРёСЏ.png"


Top
   
PostPosted: Wed Oct 31, 2018 8:47 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
I confirm it's a bug because if you remove "res://" the name is correctly dumped.
I'm investigating.
Thanks for the feedback.


Top
   
PostPosted: Wed Oct 31, 2018 9:41 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
The problem is not directly related to quickbms but in Windows not liking multiple consecutive path delimiters.

For example:
- WORKS: "res/file.png"
- DOES NOT WORK: "res//file.png" or "res\\\\\\file.png"

Basically it's a problem happening probably only the 0.1% of times but I'm happy you found it :)

It will be fixed in 0.9.1

Ah I have seen your last edit, the problem with the utf8 folder is known and will be fixed too.


Top
   
PostPosted: Wed Oct 31, 2018 11:22 pm 

Joined: Wed Oct 31, 2018 11:14 pm
Posts: 2
aluigi wrote:
Report any quickbms bug and error here.

The problems must affect quickbms and not specific scripts, for which you can use the Game Archive section.

Before reporting ANY problem be sure to have the latest version of quickbms and the latest version of the script that is causing the problem.

*edit* added tags in the title to avoid OT


Top
   
PostPosted: Thu Nov 01, 2018 5:33 am 

Joined: Sun Oct 09, 2016 6:27 pm
Posts: 28
aluigi wrote:
The problem is not directly related to quickbms but in Windows not liking multiple consecutive path delimiters.

For example:
- WORKS: "res/file.png"
- DOES NOT WORK: "res//file.png" or "res\\\\\\file.png"

Basically it's a problem happening probably only the 0.1% of times but I'm happy you found it :)

It will be fixed in 0.9.1

Ah I have seen your last edit, the problem with the utf8 folder is known and will be fixed too.

Thanks


Top
   
PostPosted: Sat Nov 17, 2018 5:42 pm 

Joined: Sat Nov 17, 2018 5:28 pm
Posts: 1
Hi, I'm trying to extract embb's from Shadow of Mordor with shadow_of_mordor.bms 0.4 and I'm getting this error.

Code:
quickbms.exe shadow_of_mordor.bms Patch_1.arch05 som

QuickBMS generic files extractor and reimporter 0.9.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Jun 17 2018 - 11:49:55)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- open input file C:\Users\limited\Downloads\quickbms\Patch_1.arch05
- open script shadow_of_mordor.bms
- set output folder som

  offset   filesize   filename
--------------------------------------
  00000000 0          bundles/player_load_skin01.bndlxml05
+ 0000304c 65536      bundles/player_load_skin01.bndlxml05

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   016E73E9
                        01210000 + 004d73e9 quickbms.exe
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    0F62EA45

Last script line before the error or that produced the error:
  183 clog NAME OFFSET CHUNK_ZSIZE CHUNK_SIZE


Code:
quickbms.exe -f "*.embb" shadow_of_mordor.bms Patch_1.arch05 som

QuickBMS generic files extractor and reimporter 0.9.0
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Jun 17 2018 - 11:49:55)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- filter string: "*.embb"
  *.embb
- filter   1: *.embb
- open input file C:\Users\limited\Downloads\quickbms\Patch_1.arch05
- open script shadow_of_mordor.bms
- set output folder som

  offset   filesize   filename
--------------------------------------
  00000000 0          bundles/player_load_skin01.embb
+ 00003c80 65536      bundles/player_load_skin01.embb

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c0000005 access violation
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   016E73E9
                        01210000 + 004d73e9 quickbms.exe
*EH* NumberParameters   00000002
*EH*                    00000000
*EH*                    0E87330F

Last script line before the error or that produced the error:
  183 clog NAME OFFSET CHUNK_ZSIZE CHUNK_SIZE



I extracted loose files from the same arch05 file with arch05explorer, but it can't extract whole embbs.
Do you still have an old shadow_of_mordor.bms script, like 0.2?


Top
   
PostPosted: Sun Nov 18, 2018 1:20 am 

Joined: Thu Aug 07, 2014 10:28 pm
Posts: 188
zstandard with the -e option worked great on most files in star ocean it fails the last chunk of this file.

never mind they used a 00 size for uncompressed data.


Attachments:
cp0409_b01a.7z [1.46 MiB]
Downloaded 5 times
Top
   
PostPosted: Mon Nov 19, 2018 1:29 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
@usernameqwerty
It looks like the identified compression is not xmemdecompress and so it crashes.
That's OT here ("not games") but feel free to post a sample file here:
viewtopic.php?f=9&t=5080&start=60

@chrrox
ok I have updated the slz script to version 0.2.5.
quickbms 0.9.1 will no longer need -e


Top
   
PostPosted: Thu Nov 22, 2018 8:51 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
0.9.1 has been released


Top
   
PostPosted: Fri Nov 23, 2018 2:10 pm 
User avatar

Joined: Sun Mar 04, 2018 1:29 am
Posts: 61
I encountered this error when extracting game's archive:

Code:
QuickBMS generic files extractor and reimporter 0.9.1
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Nov 21 2018 - 14:40:39)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                     @zenhax @quickbms  Twitter & Scripts

- GUI mode activated, remember that the tool works also from command-line
  where are available various options like folder scanning, filters and so on

- select BMS script. type ? for using the content of clipboard like a script
- select input archives/files, type * for the whole folder and subfolders
- select output folder where extracting files
- open input file C:\Users\Q.Viet\Downloads\Compressed\mod\main.136.com.ea.game.fifa14_row.obb
- open script C:\Users\Q.Viet\Downloads\Compressed\mod\fightnight.bms
- set output folder C:\Users\Q.Viet\Downloads\Compressed\mod\136

  offset   filesize   filename
--------------------------------------
  19c5fe60 182724     data/gui/locale/matchday/JPN_JP.loc
*** stack smashing detected ***:  terminated

-------------------
*EXCEPTION HANDLER*
-------------------
An error or crash occurred:

*EH* ExceptionCode      c000001d illegal instruction
*EH* ExceptionFlags     00000000
*EH* ExceptionAddress   013927A5
                        00C90000 + 007027a5 quickbms.exe
*EH* NumberParameters   00000000

Last script line before the error or that produced the error:
  92  log FNAME OFFSET SIZE

Press ENTER or close the window to quit

Sorry! But you can delete this my topic: viewtopic.php?f=9&t=8938
I was confused, you had this topic. Thank you!

_________________
Cryptography - Encryption


Top
   
PostPosted: Sat Nov 24, 2018 10:23 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
There are few users who are reporting this exact problem with "stack smashing detected" and I'm investigating.
Here on all my testing platforms (linux, mac, win98, win7, win10, win2003 and so on) I have not been able to replicate it even with the same samples used by who reported the problem, so I hope only a small number of people are affected.
I will give you some updates soon.


Top
   
PostPosted: Sat Nov 24, 2018 10:46 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9432
I will also check why 0.9.1 now takes 100 Mb of RAM more than 0.9.0 (about 280 Mb vs 180 Mb).
It's probably related to some new libraries (maybe the Capstone disassembler).
Investigating...


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 330 posts ]  Go to page Previous 113 14 15 16 17 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited