# ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
 It is currently Sat Mar 24, 2018 8:21 am

 All times are UTC

 Post new topic  Reply to topic Page 1 of 34 [ 662 posts ] Go to page 1 2 3 4 5 … 34 Next
 Print view Previous topic | Next topic
Author Message
 Post subject: Mass Effect AndromedaPostPosted: Thu Mar 16, 2017 6:42 pm

Joined: Sat Nov 05, 2016 2:19 pm
Posts: 20
I got the MEA EA and want to extract the files from it. This is the first time messing around with Frostbite 3 for me and I'm having trouble.

During my research I found the following helpful resources:

Questions:
• .toc files, wtf? What are they used for? The .sb files look more like a table of content. Are the .sb files enough to unpack?
• .toc encryption? In other games if the file is encrypted it starts with x00D1CE. In MAE it doesn't but always starts with x??D1 and ends with
Code:
B9 73 E9 D4 88 43 59 08 A5 9F 30 6A E4 D2 79 94B2 BB D1 7F 75 F4 D0 D1 01 00 00 40 65 21 61 646E 58 64 24 5E 21 72 66 4F 73 72 44 79 49 72 4921 78 56 67 48 65 41 21 36 56 63
are they encrypted? The D1 makes me think so.
• How does variable4 work? It says in the manual it's a Frostbite specific type.

I've uploaded a sample of .toc/.sb pairs here and a sample of .cas/.cat files here.

Any help is much appreciated

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Fri Mar 17, 2017 3:43 pm

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 7795
These toc files look obfuscated yes.
Two weird things: the second byte is ever 0xd1 even if the others are ever different and that 0xd0 0xd1 0x01 0x00 0x00 "@e!adnXd$^!rfOsrDyIrI!xVgHeA!6Vc" at the end. To know how the data is read in variable4 take a look at src\file.c searching "variable4": Code:  i = 0; do { c = fgetxx(fdnum, 1, error); if(*error) break; retn |= ((c & 0x7f) << i); i += 7; } while(c & 0x80); Top  Post subject: Re: Mass Effect AndromedaPostPosted: Fri Mar 17, 2017 9:44 pm Joined: Fri Mar 17, 2017 9:41 pm Posts: 14 Hey i found this but don't work for me with extraction files =), if help you =) http://files.1337upload.net/BF4_Python_ ... 18017e.rar Last edited by apollon08170 on Fri Mar 17, 2017 10:17 pm, edited 1 time in total. Top  Post subject: Re: Mass Effect AndromedaPostPosted: Fri Mar 17, 2017 10:16 pm Joined: Fri Mar 17, 2017 9:41 pm Posts: 14 When i execute bf4dumper.py with files in path Code: #This script runs through all toc files it can find and uses that information to extract the files to a target directory.#Often the assets are actually stored in cascat archives (the sbtoc knows where to search in the cascat), which is taken care of too.#The script does not overwrite existing files (mainly because 10 sbtocs pointing at the same asset in the cascat would make the extraction time unbearable).#Adjust paths here.bf4Directory = r"C:\Users\Apollon\Desktop\EXTRACTION\DATA"targetDirectory = r"C:\Users\Apollon\Desktop\EXTRACTION\extract" #do yourself a favor and don't dump into the Users folder (or it might complain about permission) ###The following paths do not require adjustments (unless the devs decided to rename their folders). Note that they are relative to bf4Directory.#As files are not overwritten, the patched files need to be extracted first.#The script will dump all tocs it can find in these two folders+subfolders:tocRoot = r"Update" #patched and xpack files FIRSTtocRoot2 = r"Data" #unpatched vanilla files SECOND#Note: The "Update" tocRoot contains both patch (for vanilla AND xpack) and unpatched xpack files. The reason it still# works correctly is because it goes through the folders alphabetically, so the patch comes first.#Feel free to comment out one or both cats if they don't exist (some Frostbite 2 games shipped without cats).#Although in that case you could just as well use an invalid path, i.e. not change anything.catPath = r"\cas.cat" updateCatPath = r"\cas.cat"................. The cas.cat file is "not found" ? Code: Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32Type "copyright", "credits" or "license()" for more information.>>> ========= RESTART: C:\Users\Apollon\Desktop\EXTRACTION\bf4dumper.py =========Unpatched cat not found.Patched cat not found.>>>  Do you have an idea ? Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 9:08 am Joined: Sun Mar 19, 2017 9:07 am Posts: 6 I tried to use the script for FIFA17, no good. Hope someone can find why! Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 3:43 pm Joined: Tue Feb 02, 2016 2:35 am Posts: 1017 I have never expected Bioware to get their hands on Frostbite 3 AGAIN. If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after youve done using it.  Attachments: frostbite.rar [3.22 KiB] Downloaded 208 times _________________ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 6:59 pm Joined: Thu Aug 13, 2015 5:08 pm Posts: 21 AnonBaiter wrote: I have never expected Bioware to get their hands on Frostbite 3 AGAIN. If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after youve done using it. Doesn't work for me, tries to open "cas_-446005142.cas" after pointing the script at a cas.cat file when the files are named cas_01.cas, cas_02.cas etc. Code: - enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage coverage file 0 0% 0 8637248 . offset 00000000- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas_-446005142.cas- error in src\file.c line 247: fdnum_open()Error: No such file or directoryLast script line before the error or that produced the error: 429 open FDSE TMP 1Press ENTER or close the window to quit Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 7:24 pm Joined: Tue Feb 02, 2016 2:35 am Posts: 1017 Oh god. Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after youre done. After all I need to fix this even if I refuse to get the game by myself. _________________ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 8:00 pm Joined: Thu Aug 13, 2015 5:08 pm Posts: 21 AnonBaiter wrote: Oh god. Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after youre done. After all I need to fix this even if I refuse to get the game by myself. Code: C:\Users\Snowpiercer\Documents\QuickBMS>quickbms -0 -V "C:\Users\Snowpiercer\Documents\QuickBMS\Scripts\frostbite_mea.bms" "C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat" "C:\Users\Snowpiercer\Documents\MEA_Extract"QuickBMS generic files extractor and reimporter 0.8.0by Luigi Auriemmae-mail: me@aluigi.orgweb: aluigi.org (Mar 12 2017 - 15:03:46) quickbms.aluigi.org Homepage zenhax.com ZenHAX Forum @zenhax Twitter & Scripts- current_folder: C:\Users\Snowpiercer\Documents\QuickBMS- bms_folder: C:\Users\Snowpiercer\Documents\QuickBMS\Scripts- exe_folder: C:\Users\Snowpiercer\Documents\QuickBMS- file_folder: C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage- output_folder: C:\Users\Snowpiercer\Documents\MEA_Extract- temp_folder: C:\Users\Snowpiercer\AppData\Local\Temp\- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat- open script C:\Users\Snowpiercer\Documents\QuickBMS\Scripts\frostbite_mea.bms- set output folder C:\Users\Snowpiercer\Documents\MEA_Extract offset filesize filename--------------------------------------. 00000000 get EXT "cat" -1002.- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage coverage file 0 0% 0 8637248 . offset 00000000- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas.cat. 0000022c idstr "NyanNyanNyanNyan" 16 4e 79 61 6e 4e 79 61 6e 4e 79 61 6e 4e 79 61 6e NyanNyanNyanNyan. 0000023c get FILES 0x0003a923 4. 00000240 getdstr ZERO "" 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 ..... 00000244 getdstr HASH "" 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 64 1f 50 .d.P. 00000258 get FOFFSET 0xd288a5cd 4. 0000025c get FSIZE 0xf45b90c6 4. 00000260 get ZERO 0x2753e5e2 4. 00000264 get PACK_NUM 0xe56a806a 4. 00000268 get PACK_NAME "cas" -1001- enter in folder C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage- open input file C:\Program Files (x86)\Origin Games\Mass Effect Andromeda\Data\Win32\streaminginstall\spinitialinstallpackage\cas_-446005142.cas- error in src\file.c line 247: fdnum_open()Error: No such file or directoryLast script line before the error or that produced the error: 429 open FDSE TMP 1 Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 8:08 pm Joined: Sun Mar 19, 2017 9:07 am Posts: 6 AnonBaiter wrote: Oh god. Try to run quickbms through command-line and use -0 -V with the script, then the file in which you want to extract and post the results here after youre done. After all I need to fix this even if I refuse to get the game by myself. Code: - open input file I:\Origin\Mass Effect Andromeda\Data\Win32\loc\en.toc- open script C:\Users\Robin-PC\Desktop\quickbms\frostbite.bms- set output folder O:\New folder offset filesize filename--------------------------------------- enter in folder I:\Origin\Mass Effect Andromeda\Data\Win32\loc coverage file 0 0% 0 115857 . offset 00000000- open input file I:\Origin\Mass Effect Andromeda\Data\Win32\loc\en.TOC- signature of 3 bytes at offset 0x00000000 doesn't match the one expected by the script: this one: "���" fd d1 e5 ... expected: "" 00 d1 ce ...- 0 files found in 0 seconds coverage file 0 0% 3 115857 . offset 00000003Press ENTER or close the window to quit Here is what I got. Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 9:11 pm Joined: Tue Feb 02, 2016 2:35 am Posts: 1017 Im working on it. Dont count on me for .sb/.toc files though. EDIT: Okay, on the frostbite.bms script, replace this line: Code:  if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" goto -16 0 SEEK_CUR endif With this line: Code:  if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" goto -16 0 SEEK_CUR else goto 0 0 SEEK_CUR endif See if it works. _________________ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Last edited by AnonBaiter on Sun Mar 19, 2017 10:06 pm, edited 1 time in total. Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 9:36 pm Joined: Thu Aug 13, 2015 5:08 pm Posts: 21 AnonBaiter wrote: Im working on it. Dont count on me for .sb/.toc files though. EDIT: Okay, on the frostbite.bms script, replace this line: Code:  if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" goto -16 0 SEEK_CUR endif With this line: Code:  if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" goto -16 0 SEEK_CUR else # do nothing endif See if it works. Getting the exact same output as before. Top  Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 9:54 pm Joined: Sat Nov 05, 2016 2:19 pm Posts: 20 aluigi wrote: These toc files look obfuscated yes. Two weird things: the second byte is ever 0xd1 even if the others are ever different and that 0xd0 0xd1 0x01 0x00 0x00 "@e!adnXd$^!rfOsrDyIrI!xVgHeA!6Vc" at the end.

To know how the data is read in variable4 take a look at src\file.c searching "variable4":
Code:
            i = 0;            do {                c = fgetxx(fdnum, 1, error);                if(*error) break;                retn |= ((c & 0x7f) << i);                i += 7;            } while(c & 0x80);

Thanks, how can I deobfuscate the .toc files? And can you tell me what is in them that isn't in the .sb files?

apollon08170 wrote:
Hey i found this but don't work for me with extraction files =), if help you =)

I checked that out thank you.

AnonBaiter wrote:
I have never expected Bioware to get their hands on Frostbite 3 AGAIN.

If you want to dump just the .cas/.cat files then use my modified script. I want to know if it works on this "game" after youve done using it.

Can you tell me how to save the bundles? What's up with ebx, dbx, res, chunk, bmm?
And can you tell me what's usually in the .toc files? And why is there so many duplicates?

I can read cas.cat files and .sb files with the attached project, it's a VS C# program. Currently working on writing the bundles to disk.

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 10:12 pm

Joined: Tue Feb 02, 2016 2:35 am
Posts: 1017
swinei wrote:
Can you tell me how to save the bundles? What's up with ebx, dbx, res, chunk, bmm?
And can you tell me what's usually in the .toc files? And why is there so many duplicates?
What? I don't have the game, and I won't intend to have it anytime soon. You don't want to know why.

_________________
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 10:22 pm

Joined: Sat Nov 05, 2016 2:19 pm
Posts: 20
AnonBaiter wrote:
What? I don't have the game, and I won't intend to have it anytime soon. You don't want to know why.

Now I'm really curios why not?

You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at. I can also upload more sample files if you need anything specific.

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Sun Mar 19, 2017 11:06 pm

Joined: Tue Feb 02, 2016 2:35 am
Posts: 1017
swinei wrote:
You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at.
Well, to be honest though I'm just a beginner on this stuff.
swinei wrote:
I can also upload more sample files if you need anything specific.
Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

_________________
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Mon Mar 20, 2017 3:41 am

Joined: Sun Mar 19, 2017 9:07 am
Posts: 6
AnonBaiter wrote:
swinei wrote:
You seem to know the Frostbite engine that's why I ask. I don't have any other Frostbite games to look at.
Well, to be honest though I'm just a beginner on this stuff.
swinei wrote:
I can also upload more sample files if you need anything specific.
Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

So is there any possible to extract files from toc and sb files?

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Mon Mar 20, 2017 3:54 am

Joined: Tue Feb 02, 2016 2:35 am
Posts: 1017
No ideas at the moment.

_________________
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Mon Mar 20, 2017 7:45 am

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 7795
AnonBaiter wrote:
if ZERO != "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"[/code]See if it works.

That can't work because you can't use C escapes in conditions.
Strings containing escapes are allowed only in those few commands where it's written "cstring" in quickbms.txt (unfortunately this is an "heritage" of the original bms language).

If TOC is encrypted there is no way to extract data from it using the current script.

Top
 Post subject: Re: Mass Effect AndromedaPostPosted: Tue Mar 21, 2017 8:28 am

Joined: Sat Nov 05, 2016 2:19 pm
Posts: 20
The CAS entries are compressed with Zstandard. Yay comptype scanner
compression type = 0F 70

The resId for textures is 1809719482

So now I have the res entry with the DDS header info and the chunk entry with the texture data.

The DDS header info has the 16 byte chunk id at position 32.
The last 16 bytes is maybe the name, but multiple entries can have the same name hm.
Width and height shorts start at 22 (I think).

I've attached an example pair, any help figuring out the header would be much appreciated!

edit: I started on a 010 template with info from here and here
Code:
//------------------------------------------------//--- 010 Editor v7.0.2 Binary Template////      File: //   Authors: //   Version: //   Purpose: //  Category: // File Mask: //  ID Bytes: //   History: //------------------------------------------------int mipOneEndOffset;int mipTwoEndOffset;int textureType;int pixelFormat;short unknown;short depth;short sliceCount;short width;short height;byte numSizes;byte firstMip;int dummy;char chunkID[16];local int i;for(i=0;i<15;i++){    int mipSize;}int chunkSize;byte dummyy[16];int nameHash;char name[16];

AnonBaiter wrote:
Well, to be honest though I'm just a beginner on this stuff.

Ha me too
AnonBaiter wrote:
Thanks for the offer, but I already have a Frostbite 3 game on my HDD. Which is enough since my Frostbite "research" is limited to games made by EA DICE with that engine. That habit of mine might change sometime though, just not today.

Could you upload some decrypted .toc files please? I wanna know whats in them