ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Tue Oct 16, 2018 2:43 pm

All times are UTC




Post new topic  Reply to topic  [ 26 posts ]  Go to page 1 2 Next
Author Message
PostPosted: Fri Oct 05, 2018 1:49 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Okay so first things first, yes i tried the bms script made for the Division and yes it mostly works but there are issues.

Basically, it eventually runs into an error like this:
Code:
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-C-2265.sdfdata
  00000000 119        moria/baked/art/[assets]/[generictexture]/gn_roof_tile_bitumen_01_d.dds

- The following output file already exists:
  moria\baked\art\[assets]\[generictexture]\gn_roof_tile_bitumen_01_d.dds
  Do you want to overwrite it?
    y = overwrite (you can use also the 'o' key)
    n = skip (default, just press ENTER)
    a = overwrite all the files without asking
    r = automatically rename the files with the same name
    s = skip all the existent files without asking



Some files are split across the SDFDATA archives and the script runs into issues with this, but
no matter what option you choose here it immediately goes into this:
Code:
Error: incomplete input file 1: C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-C-2265.sdfdata
       Can't read 114 bytes from offset 00000005.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1   100%   5          5          . offset 00000005

Last script line before the error or that produced the error:
  127 log name packageOffset decompressedSize 1


The script also doesn't support the merging of DDS headers from the SDFTOC file to the output DDS files in the extracted SDFDATA, so it has to be done manually at the moment, Although theres a whole other issue with those that i'd need to go post about on the Graphics Files board instead(Basically its an obscure DXGI format that uses ASTC i think)

Running the script on smaller amounts of SDFDATA at a time seems to help with the issues a bit but the file splitting is still an issue.
One of my friends was trying to modify the tool the script is based off (Rogue_SDF) to work with this game but ran into issues with "chunks"? That's all i know.
I'll send samples if required but i think a full dump may be needed to properly investigate the issues.

Any help would be appreciated :)
In the meantime here is a screenshot of one of the models that managed to extract properly and load in Noesis.
Image


Top
   
PostPosted: Fri Oct 05, 2018 4:35 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
Do you mean you have already tried the http://aluigi.org/bms/ubisoft_sdf.bms script?


Top
   
PostPosted: Fri Oct 05, 2018 4:50 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Yes, that's the one.


Top
   
PostPosted: Fri Oct 05, 2018 5:05 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
Could you please upload the sdftoc file?
Basically sdftoc is telling you that sdf-C-2265.sdfdata contains a file with a duplicated name and a size of 114 bytes while that "container" is 5 bytes only, so there is something weird or something else to implement.


Top
   
PostPosted: Fri Oct 05, 2018 5:36 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
https://cdn.discordapp.com/attachments/419711036837330956/497824314033438723/sdf.sdftoc
It's zstd compressed if i remember correctly.


Top
   
PostPosted: Fri Oct 05, 2018 6:09 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
Long story short, I don't know what's the problem.

Everything is correct till the code "for chunkIndex = 0 < count1" reads ch3 from offset 0x0001a560 of MEMORY_FILE which is 0x00, and this is the only ch3 with this value.
So what I did was ignoring the instructions if ch3 is 0x00 (which is an invalid value indeed) and apparently everything works correctly :)
Script 0.1.3a, let me know if you get other errors.


Top
   
PostPosted: Fri Oct 05, 2018 6:19 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Same errors as before but it happens slightly later i think.


Top
   
PostPosted: Fri Oct 12, 2018 2:19 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Okay, i forgot about another error that happens if it doesn't run into any existing files. .
Code:
Error: incomplete input file -1:
       Can't read 3 bytes from offset 00527172.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file -1  117%   6355722    5402994    . offset 00527172

Last script line before the error or that produced the error:
  87  get fileId long MEMORY_FILE


Top
   
PostPosted: Fri Oct 12, 2018 5:00 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
Yesterday I updated the script for supporting version 0x17 and it may fix that error.
Anyway don't worry since it just means you have extracted everything and you can ignore it.


Top
   
PostPosted: Fri Oct 12, 2018 7:54 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
The issue with files split across archives is still there :(


Top
   
PostPosted: Sat Oct 13, 2018 8:35 am 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
The other issue is gone, forgot to mention that somehow.
.


Top
   
PostPosted: Sat Oct 13, 2018 10:05 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
Can you remind me what's the issue with "files split across archives"?
The script supports the various sdfdata and I'm not aware of files that are located half in one and half in another archive.


Top
   
PostPosted: Sat Oct 13, 2018 11:38 am 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Well one would be Mario's model, i think.

Code:
- enter in folder C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-A-0208.sdfdata
  00000000 0          moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
+ 0001b283 65536      moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
+ 00021b5c 65536      moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
+ 000294fa 30365      moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb
- enter in folder C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data
  coverage file 1     0%   72403      24936897   . offset 00000000
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-A-0209.sdfdata
  000092f9 2230304    moria/baked/art/animation/01-chr/02-her/mario/chr_her_mario_tpose_01.mmb

- The following output file already exists:
  moria\baked\art\animation\01-chr\02-her\mario\chr_her_mario_tpose_01.mmb
  Do you want to overwrite it?
    y = overwrite (you can use also the 'o' key)
    n = skip (default, just press ENTER)
    a = overwrite all the files without asking
    r = automatically rename the files with the same name
    s = skip all the existent files without asking



And no matter what you choose this happens:
Code:
Error: incomplete input file 1: C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-D-3846.sdfdata
       Can't read 10341 bytes from offset 00000005.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1   100%   5          5          . offset 00000005

Last script line before the error or that produced the error:
  155 clog name packageOffset compSize CHUNK_SIZE 1

Press ENTER or close the window to quit


Top
   
PostPosted: Sat Oct 13, 2018 11:44 am 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
SDFDATA 3846 seems to be a 5 byte dummy file that apparently contains 2 parts of a model???
Something is really odd here.
Code:
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-D-3846.sdfdata
  00000000 0          moria/baked/art/animation/01-chr/03-enm/globetrotter/chr_enm_shd_globetrotter_tpose_01.mmb
+ 00000000 65536      moria/baked/art/animation/01-chr/03-enm/globetrotter/chr_enm_shd_globetrotter_tpose_01.mmb


Image


Top
   
PostPosted: Sat Oct 13, 2018 11:50 am 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Removing all the dummy files still ends up with a similar issue,
Code:
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-B-1702.sdfdata
  00c79b12 54030      moria/baked/art/animation/01-chr/03-enm/shielder/chr_enm_lvp_shielder_tpose_01.mmb
- enter in folder C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data
  coverage file 1     0%   24139      25137233   . offset 00000000
- open input file C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-B-1703.sdfdata
  000081da 12257012   moria/baked/art/animation/01-chr/03-enm/shielder/chr_enm_lvp_shielder_tpose_01.mmb

Error: incomplete input file 1: C:\Users\faint\Documents\SwitchDumps\MarioRabbidsKingdomBattle\romfs\moria\sdf\nx\data\sdf-B-1703.sdfdata
       Can't read 101529 bytes from offset 008ef541.
       Anyway don't worry, it's possible that the BMS script has been written
       to exit in this way if it's reached the end of the archive so check it
       or contact its author or verify that all the files have been extracted.
       Please check the following coverage information to know if it's ok.

  coverage file 1    99%   9335655    9368897    . offset 008ef541

Last script line before the error or that produced the error:
  140 log name packageOffset decompressedSize 1

Press ENTER or close the window to quit


Top
   
PostPosted: Sat Oct 13, 2018 2:16 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
I already provided a full explanation of why it's impossible to extract N bytes from a dummy archive containing only 5 bytes.


Top
   
PostPosted: Sat Oct 13, 2018 2:43 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 9043
Let's say that ch3 equal than zero should be handled as a "break" instead of a "continue" (like it's right now).
I have updated the script for trying this other method and fixing an unrelated bug in DUMP_FILE.
Let me know if the situation is now better.


Top
   
PostPosted: Sat Oct 13, 2018 3:42 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Still getting the
Quote:
CHUNK_SIZE 1
error from it trying to extract from a dummy file, so no change from the previous version basically.

Very confusing issue...
I'll upload a batch of samples shortly. (i already have the entire game uploaded to google drive so if the samples aren't enough help i can pm a link to that or something, don't wanna link something like that publicly)


Top
   
PostPosted: Sat Oct 13, 2018 3:52 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
Alright here's the samples.
There was a file in a folder called chunkmapping.mchunks at the
Quote:
romfs\moria\sdfconfig\nx
directory that i thought i should include.
Everything else is from the
Quote:
romfs\moria\sdf\nx\data
directory, the sdfver file is probably useless but i included it anyway.
https://drive.google.com/file/d/1HSKLQl ... sp=sharing

EDIT: Of course i forgot the link.


Top
   
PostPosted: Sat Oct 13, 2018 4:09 pm 

Joined: Thu Sep 13, 2018 6:38 pm
Posts: 43
https://cdn.discordapp.com/attachments/419711036837330956/500701420807258113/sdf-slice.mapping
This might be helpful too.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 26 posts ]  Go to page 1 2 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited