ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sat Feb 25, 2017 1:47 pm

All times are UTC




Post new topic  Reply to topic  [ 20 posts ] 
Author Message
PostPosted: Wed Oct 29, 2014 8:02 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Edit: I've worked out the following script to extract data from WADs. It's not perfect and ignores all grouping/HEAP data, but it' most of the way there. Works on GoW1/2 for both PS2 and PS3, and GoWA. Untested with GoW3.

Thanks for the help aluigi.

Note: Skips CAT 0x18 to avoid deciphering HEAP in R_PERM.
Code:
get ARCHIVE_SIZE asize
get DUMMY long
get ENDTEST short

if ENDTEST == 0
    endian big
    math ENDIPAD = 0x20
endif

for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

    math OFFSET += 0x20
    math OFFSET += ENDIPAD

    if NAME == "EntityCount" || CAT == 0x18
        math SIZE = 0
   math valid = 0
    endif

    if valid != 0 && size > 0
        log NAME OFFSET SIZE
    endif

    math OFFSET += SIZE
    math OFFSET x= 0x10
next


List of non-standard WADs not covered by this script:
GoW2 - R_mcicon.wad_ps3
GoW3 - R_NETWORK.WAD
GoWA - R_MULTIPLAYER.WAD


Last edited by Wulf on Sun Nov 16, 2014 10:05 pm, edited 8 times in total.

Top
   
PostPosted: Thu Oct 30, 2014 5:05 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Edit: Removed.


Last edited by Wulf on Fri Nov 07, 2014 5:45 am, edited 1 time in total.

Top
   
PostPosted: Fri Oct 31, 2014 4:19 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
From GoW1-PS3-r_shell.trimmed.wad_ps3:

At 0x1c1f8 there is the value 00 0A AA AC.
At 0x1c264 32-bit pixel data starts and carries on until 0xc6d10, which is 0xaaaac bytes long.




------

There are 180 "GroupStart"s, and only 122 "GroupEnd"s.


Top
   
PostPosted: Sat Nov 01, 2014 4:30 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5524
I guess that the thread you opened in 2012 didn't give results, correct?

The format doesn't seem to be so simple to parse.


Top
   
PostPosted: Tue Nov 04, 2014 3:42 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Quote:
I guess that the thread you opened in 2012 didn't give results, correct?
Correct.

The WADs from the PS3 versions seem a little easier to understand because they have recognizable data in them (DDS, BMP, etc), but aside from picking out individual files from the data I haven't been able to get an idea of its overall layout.


Top
   
PostPosted: Tue Nov 04, 2014 8:00 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5524
Yeah, it "seems" simple but it isn't.
I tried to make a very quick and simple test but it failed:
Code:
endian big
math OFFSET = 0xd00
for
    goto OFFSET
    padding 0x10
    get DUMMY short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x38
    get DUMMY short
    get DUMMY short
    getdstring NAME 0x38
    getdstring DUMMY 0x14
    savepos OFFSET
    math SIZE -= 0x50
    log NAME OFFSET SIZE
    math OFFSET += SIZE
next

Anyway the fact that it uses normal files it's good, you can try to rip them with DragonUnpacker (it has a simple ripper embedded as far as I remember).


Top
   
PostPosted: Tue Nov 04, 2014 10:17 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Might take a crack at attaching a debugger and watching the game parse the WAD, but it'll probably be beyond my understanding of PPC.

Edit: Dragon Unpacker finds a single, giant DDS file in the GoWA Shell. Nothing in the others.


Top
   
PostPosted: Wed Nov 05, 2014 11:27 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5524
Very strange the behaviour of Dragon Unpacker.
A ripper should rip everything, in fact there are tons of DDS in the sample you provided.
There are other file rippers but they are dead/old or maybe not safe.


Top
   
PostPosted: Wed Nov 05, 2014 3:09 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
The following could be incorrect in minor ways, it was late when I looked at it and I wasn't documenting. I'll take a better look tonight.

The game starts off by loading the entire WAD into memory without parsing it. Once it's in memory, one function will read up to "PopHeap" before passing it off.

The GroupStart/GroupEnd lines are read in 0x20 byte chunks, starting at even 0x10 multiples of the WAD file. Meaning, the following would be a single read:
Code:
28 00 00 00 00 00 00 00 47 72 6F 75 70 53 74 61     (.......GroupSta
72 74 00 00 00 00 00 00 00 00 00 00 00 00 00 00     rt..............


The xxxX_R_Shell lines are read in 0x30 byte chunks, after the endianness of 0x20-0x24 is reversed. Meaning, the red bytes would be reversed, then the chunk would be read:
1E 00 00 00 30 00 00 00 47 46 58 58 5F 52 5F 53 ....0...GFXX_R_S
68 65 6C 6C 00 00 00 00 00 00 00 00 00 00 00 00 hell............
0C 00 00 80 00 00 00 00 00 04 00 00 00 04 00 00 ...€............

After hitting PopHeap, it would pass it off to another function which would already know the start and end bytes of the data it cared about, and read it in up to 0x20000 byte chunks. I have to trace that one back to see where it's calculating those.


Top
   
PostPosted: Wed Nov 05, 2014 9:37 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Edit: Removed


Last edited by Wulf on Fri Nov 07, 2014 4:01 pm, edited 1 time in total.

Top
   
PostPosted: Thu Nov 06, 2014 6:54 am 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Well, I'm successfully using a hex editor to rip and replace models in the game without it crashing. Not with 100% success, but a bunch of weird orbs running around in place of the player is a start.

Looks like model data changed between GoW1 and GoW2, no success swapping the Kratos costumes between games.


Top
   
PostPosted: Thu Nov 06, 2014 12:36 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5524
Maybe you can try to put everything in a script to automatize the extraction.
When you understand the format, the script is simple.


Top
   
PostPosted: Thu Nov 06, 2014 5:12 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Having a bit of trouble with the file ending offset seeming to be inconsistent... As in, jumping ahead by the amount shown will sometimes land me partway through the next header.

Also haven't fully figured out how to deal with GroupStart/Ends. I assume it relates to the WAD being treated as a heap, but I don't fully understand the concept as it would apply to an archive file.


Top
   
PostPosted: Thu Nov 06, 2014 6:26 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Edit: Moved to first post.


Optional insert:
Code:
string NAME p= "%08X-%s" OFFSET NAME

More work needs to be done to figure out how the grouping works, but this seems to extract the pieces.

Works on GoW 1 and 2 Shell WADs so far.


Last edited by Wulf on Fri Nov 07, 2014 2:43 pm, edited 3 times in total.

Top
   
PostPosted: Thu Nov 06, 2014 10:32 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Edit: Removed, combined with script in first post.


Last edited by Wulf on Fri Nov 07, 2014 4:01 pm, edited 1 time in total.

Top
   
PostPosted: Fri Nov 07, 2014 11:22 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 5524
Well done, I have seen you made also a video.

I made just some small aestethic changes to your script, only to make (I hope) a bit more readable.
Maybe it's useful to know how to tweak the scripts for doing the same job with less instructions and to use indentation, the only thing I changed was "SIZE == 0x50" with "SIZE <= 0x50" (is it better?).
Code:
endian big
get ARCHIVE_SIZE asize
for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

    math OFFSET += 0x40

    if NAME == "EntityCount"
        math SIZE = 0x20
    elif NAME == "GoStreamInfo"
        math SIZE = 0x50
    endif

    if SIZE <= 0x50
        math valid = 0
    endif

    if valid != 0
        log NAME OFFSET SIZE
    endif

    math OFFSET += SIZE
    math OFFSET x= 0x10
next


Should this script work also with the sample file you provided?


Top
   
PostPosted: Fri Nov 07, 2014 4:42 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
That was a good starting off point. I tweaked some of it around, and the one script now works with GoW1/2 and GoWA. I'll dig out my disc later tonight and test with GoW3.

I'm pretty happy with it so far but still need to figure out the significance of GroupStart/End. Being able to throw all the models/textures/scripts/sounds into designated folders would make everything much simpler.


Top
   
PostPosted: Fri Nov 07, 2014 10:07 pm 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Alternate version of the script, to output files numbered in the order they're processed. Also outputs empty GroupStart/End files for analysis.
Code:
get ARCHIVE_SIZE asize
get DUMMY long
get ENDTEST short

if ENDTEST == 0
    endian big
    math ENDIPAD = 0x20
endif

math count = 1

for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

    string NAME p= "%04d-%s" COUNT NAME

    math OFFSET += 0x20
    math OFFSET += ENDIPAD

    if NAME == "EntityCount" || CAT == 0x18
        math SIZE = 0
   math valid = 0
    endif


        log NAME OFFSET SIZE


    math OFFSET += SIZE
    math OFFSET x= 0x10
    math COUNT += 1
next


Top
   
PostPosted: Sun Nov 16, 2014 5:13 am 

Joined: Mon Oct 27, 2014 8:30 pm
Posts: 46
Modified version to allow greater success with GoWA WADs.

Will be put into the first post once I pretty it up and confirm I didn't break the other GoWs.

Code:
get ARCHIVE_SIZE asize
get DUMMY long
get ENDTEST short

if ENDTEST == 0
    endian big
    math ENDIPAD = 0x20
endif

math count = 1

for OFFSET = 0 < ARCHIVE_SIZE
    math valid = 1

    goto OFFSET
    get CAT short
    get DUMMY short
    get SIZE long
    getdstring NAME 0x18

   

    math OFFSET += 0x20
    math OFFSET += ENDIPAD

    if NAME == "EntityCount" || CAT == 0x18
        math SIZE = 0
        math valid = 0
    endif

  if NAME == "N_SCREENS" || NAME == "DELAY_00"
    math size = 0
  endif

  if NAME == "MSGS_COUNT" || NAME == "MSGS_LINES"
    math size = 0
  endif

  if NAME == "HERO_HEAP_SIZE" || NAME == "UPGRADE_HEAP_SIZE"
    math size = 0
  endif

  if NAME == "ContextNames"
    math valid = 0
  endif


if VALID == 1
  string NAME p= "%s.%06d" NAME COUNT
  log NAME OFFSET SIZE
endif

    math OFFSET += SIZE
    math OFFSET x= 0x10
    math COUNT += 1
next


Top
   
PostPosted: Wed Mar 23, 2016 5:19 am 

Joined: Wed Mar 23, 2016 5:11 am
Posts: 2
Hey guys, I know its an old thread, but was wondering, was anyone ever fully able to extract proper texture/meshes/animation from any of the GoW series?
Tried all 4 samples of the script here, on all four PS3 games, but there is a mess up on extraction, is there anything updated and usable?

cheers.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 20 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited