ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Mon Oct 23, 2017 2:40 am

All times are UTC




Post new topic  Reply to topic  [ 28 posts ]  Go to page 1 2 Next
Author Message
 Post subject: FIFA 17
PostPosted: Tue Sep 27, 2016 5:35 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
The bad news is that FIFA 17 is not supported yet.

The new format is made just to complicate the extraction with TOC which is a reference to SB (as usual) and SB a reference for the CAT files that are a reference to the CAS archives... yeah: TOC->SB->CAT->CAS

I found a way to implement this complex reference in my frostbite.bms script but it's experimental and doesn't cover the compression used for the chunked files, I don't know what compression it uses.

The updated and totally useless script is 0.3:
http://aluigi.org/bms/frostbite.bms

If you select the TOC/SB files: the script will locate and dump the raw compressed files indexed there from CAS
If you select the CAT/CAS files: the script will make a raw nameless dump of everything is contained in the archives (better imho)

I'm not so interested in figuring the compression at the moment or continuing with this complex toc/sb/cat/cas format, so I provide the script "as-is".


Top
   
 Post subject: Re: FIFA 17
PostPosted: Tue Sep 27, 2016 8:33 pm 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 969
Nice effort. Do you know if this script works also for other Frostbite games such as ones that aren't listed on the header of the script?

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Sep 28, 2016 8:21 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
It should but it's better if you provide some samples (toc is probably enough) because this FIFA17 uses version 1 and I don't want that it collides with other versions 1 that don't use cat/cas


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Sep 28, 2016 11:52 am 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 969
Someday I will.

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Sep 28, 2016 7:11 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
In case someone wants to check the compression used by the archives, I have attached a full chunk (full file too) with a compressed size of 0x07e1 bytes and a decompressed size of 0x1088.


Attachments:
compressed_chunk.dat [1.98 KiB]
Downloaded 81 times
Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 05, 2016 2:45 am 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 969
Well, if you really claim that just the .sb/.toc pairs are enough, then here they are:
https://mega.nz/#F!fNdxDBJJ!5MIMEp_JOGIbuc23iTp2Zg

These files came from Mirror's Edge Catalyst.

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 05, 2016 9:01 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
It seems compatible.
The only problem remains the compression algorithm that I hope to solve one day.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 05, 2016 1:00 pm 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 969
Except not. This is what happened when I loaded the script with an .cat file:
Code:
QuickBMS generic files extractor and reimporter 0.7.6a (64bit test)
by Luigi Auriemma
e-mail: me@aluigi.org
web:    aluigi.org
        (Aug 22 2016 - 18:02:49)

                   quickbms.aluigi.org  Homepage
                            zenhax.com  ZenHAX Forum
                               @zenhax  Twitter & Scripts

- current_folder: E:\quickbms
- bms_folder:     E:\quickbms_scripts
- exe_folder:     E:\quickbms
- file_folder:    E:\3DMGAME-Mirrors.Edge.Catalyst-CPY\extracted\MirrorsEdgeCatal\Setup\Data\Win32\gameconfigurations\defaultinstallpackage
- output_folder:  E:\quickbms
- temp_folder:    C:\Users\TCA\AppData\Local\Temp\
- open input file E:\3DMGAME-Mirrors.Edge.Catalyst-CPY\extracted\MirrorsEdgeCatal\Setup\Data\Win32\gameconfigurations\defaultinstallpackage\cas.cat
- open script E:\quickbms_scripts\frostbite.bms
- set output folder .

  offset           filesize   filename
--------------------------------------
. 0000000000000000 get     EXT        "cat" -1002
.
- enter in folder E:\3DMGAME-Mirrors.Edge.Catalyst-CPY\extracted\MirrorsEdgeCatal\Setup\Data\Win32\gameconfigurations\defaultinstallpackage
  coverage file 0     0%   0          569632
- open input file E:\3DMGAME-Mirrors.Edge.Catalyst-CPY\extracted\MirrorsEdgeCatal\Setup\Data\Win32\gameconfigurations\defaultinstallpackage\cas.cat
. 000000000000022c idstr              "NyanNyanNyanNyan" 16
    4e 79 61 6e 4e 79 61 6e 4e 79 61 6e 4e 79 61 6e   NyanNyanNyanNyan
. 000000000000023c get     FILES      0x0000000000003dbf 4
. 0000000000000240 getdstr ZERO       "" 20
    00 00 00 00 72 f3 61 61 a4 99 81 1e 7e b0 a0 6c   ....r.aa....~..l
    54 b0 c2 0f                                       T...
. 0000000000000254 getdstr HASH       "" 20
    d6 28 5d c3 00 00 00 00 7b 0f 00 00 00 00 00 00   .(].....{.......
    01 00 00 00                                       ....
. 0000000000000268 get     FOFFSET    0x00000000f9d95eb1 4
. 000000000000026c get     FSIZE      0x00000000761844a0 4
. 0000000000000270 get     ZERO       0x00000000740655c2 4
. 0000000000000274 get     PACK_NUM   0x00000000fdf21193 4
. 0000000000000278 get     PACK_NAME  "cas" -1001
- enter in folder E:\3DMGAME-Mirrors.Edge.Catalyst-CPY\extracted\MirrorsEdgeCatal\Setup\Data\Win32\gameconfigurations\defaultinstallpackage
- open input file E:\3DMGAME-Mirrors.Edge.Catalyst-CPY\extracted\MirrorsEdgeCatal\Setup\Data\Win32\gameconfigurations\defaultinstallpackage\cas_-34467437.cas

- error in src\file.c line 243: fdnum_open()
Error: No such file or directory

Last script line before the error or that produced the error:
  419 open FDSE TMP 1
The same error also happened with a .sb/.toc file(report attached). But anyway, this is the error:
Code:
get FOFFSET long 1
Yep.


Attachments:
default.rar [2.26 MiB]
Downloaded 77 times

_________________
Expect my username to be everywhere, whether you're using the internet or not.
Top
   
 Post subject: Re: FIFA 17
PostPosted: Thu Oct 06, 2016 12:15 am 
User avatar

Joined: Tue Feb 02, 2016 2:35 am
Posts: 969
I updated the MEGA folder with a .cas/.cat sample file, if you don't mind. Or do you have the files already?

_________________
Expect my username to be everywhere, whether you're using the internet or not.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Thu Oct 06, 2016 8:39 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
I'm going to buy the game but I'm not sure when I will check this stuff, boring stuff, better to play :)


Top
   
 Post subject: Re: FIFA 17
PostPosted: Mon Oct 10, 2016 12:17 pm 

Joined: Mon Oct 10, 2016 11:43 am
Posts: 5
Hi Friends, How are you?

First of all I wanted to thank you, aluigi for the effort. You did a wonderful job with your Scripts and it's remarkable because you just did to help the community. I'm kinda newbie in all this, the fact is that I reached here by a coincidence, jumping from a website to another, but found it fascinanting.

I'm owner, like many people of a FIFA 17 Polish edition (50% half the prize....who wouldn't do the same?) and like another years, I wanted to fix the language. I'm not sure if you know it, but was very easy fixing the language. EA used until this year .big files that contained xml tables and files easily editable. But this year, they changed the file system. They seem to use something similar to BF4 and another Frostbite games. They have .cas files that I suspect that contains the "heavy" data and some .toc and .sb files. I think that .toc files are the key, so I tried to read them with hex editors. They were crypted and then I began to search how could I decrypt and read them.

I have to confess that I could just simply forgot and wait until someone fix it, I'm just multiplayer guy and for me it's same. But I have that fever for Knowledge, that hunger for solving problems that I know many of you could understand. I'm java programmer and I'd love to learn how to solve this.

Until now, I tried to use your Frostbite Script with the pol.toc and pol.sb files, but don't know why it don't find any files. I upload the files. What should i do or aim for?. I'd love any possible tip or explanaition because as I said, I'm lost.

Thanks in advance.

Attachment:
screenshoot1.jpg [376.56 KiB]
Not downloaded yet

Attachment:
screenshoot2.jpg [468.8 KiB]
Not downloaded yet


Top
   
 Post subject: Re: FIFA 17
PostPosted: Tue Oct 11, 2016 10:06 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
Regarding this script I think it's probably not ready yet.
It works with the FIFA17 demo, don't know why it doesn't work with the full game.
I still have to test/adapt it to Mirror's Edge but, honestly, I don't care much at the moment and prefer to use my free time to jump on the buildings of City of Glass :D


Top
   
 Post subject: Re: FIFA 17
PostPosted: Tue Oct 11, 2016 5:30 pm 

Joined: Tue Oct 11, 2016 5:10 pm
Posts: 4
Hey folks!

I've updated my personal collection of Frostbite dumping scripts (which are mostly based on Frankelstner's original scripts, but heavily modified over the years for completeness' sake) in order to make them work with FIFA 17, and I thought I'd share them with you.

They can currently successfully extract and decompress any CAS-based content, including ebx files, resources, and chunks, and also fully process and dump ebx files in the usual text format one can expect from Frank's scripts. Please note that they currently do not support non-CAS based content (such as self-contained SuperBundle content or patch data), and that they have only been tested to successfully work with the FIFA 17 Demo.

You can grab the scripts from here.

In order to extract your game's data make sure you have the 64-bit version of Python 2.7 installed, extract the scripts somewhere, and then adjust FB2Dumper.py and ebx.py so that they point to your game directory. Afterwards, simply run python FB2Dumper.py followed by python ebx.py (after FB2Dumper is finished) in your command-prompt and you should be set to go. Please note that the ebx dumping script can (and probably will) use up all of your Disk I/O and quite some CPU.

If you encounter any issues feel free to let me know, but keep in mind that I don't have the full game at my disposal, so any issues related to that will be relatively hard for me to resolve.

Cheers,
NoFaTe


Last edited by NoFaTe on Tue Oct 11, 2016 9:43 pm, edited 2 times in total.

Top
   
 Post subject: Re: FIFA 17
PostPosted: Tue Oct 11, 2016 5:37 pm 

Joined: Mon Oct 10, 2016 11:43 am
Posts: 5
aluigi wrote:
Regarding this script I think it's probably not ready yet.
It works with the FIFA17 demo, don't know why it doesn't work with the full game.
I still have to test/adapt it to Mirror's Edge but, honestly, I don't care much at the moment and prefer to use my free time to jump on the buildings of City of Glass :D


Nice, I will try to use with the demo in order to learn how it works. What's your opinion about the language change of FIFA?. It's in the .toc file or .cas?.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Tue Oct 11, 2016 7:23 pm 

Joined: Mon Oct 10, 2016 11:43 am
Posts: 5
NoFaTe wrote:
Hey folks!

I've updated my personal collection of Frostbite dumping scripts (which are mostly based on Frankelstner's original scripts, but heavily modified over the years for completeness' sake) in order to make them work with FIFA 17, and I thought I'd share them with you.

They can currently successfully extract and decompress any CAS-based content, including ebx files, resources, and chunks, and also fully process and dump ebx files in the usual text format one can expect from Frank's scripts. Please note that they currently do not support non-CAS based content (such as self-contained SuperBundle content or patch data), and that they have only been tested to successfully work with the FIFA 17 Demo.

You can grab the scripts from here.

In order to extract your game's data make sure you have the 64-bit version of Python 2.7 installed, extract the scripts somewhere, and then adjust FB2Dumper.py and ebx.py so that they point to your game directory. Afterwards, simply run python FB2Dumper.py followed by python ebx.py (after FB2Dumper is finished) in your command-prompt and you should be set to go. Please note that the ebx dumping script can (and probably will) use up all of your Disk I/O and quite some CPU.

If you encounter any issues feel free to let me know, but keep in mind that I don't have the full game at my disposal, so any issues related to that will be relatively hard for me to resolve.

Cheers,
NoFaTe


I'm having troubles running your script I've got constant mistakes of Syntax, I installed several versions of Python 2.7. If you can paste me the link to the version you use I'd be glad.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Tue Oct 11, 2016 8:05 pm 

Joined: Tue Oct 11, 2016 5:10 pm
Posts: 4
Grieftime wrote:
I'm having troubles running your script I've got constant mistakes of Syntax, I installed several versions of Python 2.7. If you can paste me the link to the version you use I'd be glad.


Make sure the Python versions you have installed do not conflict with each other (you can tell by just running python and seeing what the first line says).
In any case, here's the exact version I'm using, but any 64-bit version of Python 2.7.x should work.
If you're still getting syntax errors please paste your traceback so I can see what's causing them.

Edit: I realized I forgot to remove some of my testing code which caused some files to not dump. You can find an updated version here.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 12, 2016 6:12 am 

Joined: Mon Oct 10, 2016 11:43 am
Posts: 5
NoFaTe wrote:
Grieftime wrote:
I'm having troubles running your script I've got constant mistakes of Syntax, I installed several versions of Python 2.7. If you can paste me the link to the version you use I'd be glad.


Make sure the Python versions you have installed do not conflict with each other (you can tell by just running python and seeing what the first line says).
In any case, here's the exact version I'm using, but any 64-bit version of Python 2.7.x should work.
If you're still getting syntax errors please paste your traceback so I can see what's causing them.

Edit: I realized I forgot to remove some of my testing code which caused some files to not dump. You can find an updated version here.



Thank you. I'll give them a try but they seem to run now. I'll study them more detailed later. I don't have experience with Python scripts so It's hard for me understanding some of the syntax. Anyways as programmer I understand the structures, callings and all that.


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 12, 2016 8:09 am 

Joined: Mon Oct 10, 2016 11:43 am
Posts: 5
I tried them and I face some troubles. FB2Dumper seems to work, It executes and dump, but I don't see any output. I'll add the screens of the FB2Dumper with modified paths and the console result. I'm not sure If i did something wrong. After running the FB2Dumper.py I load the ebx and I run it, but IT doesn't find any file, this is the result.

Attachment:
FB2Dumper.jpg [196.5 KiB]
Not downloaded yet


Attachment:
FB2Dumper_console.jpg [269.53 KiB]
Not downloaded yet


What should I specify in:
inputFolder = r"H:\Research\FIFA17\Dump\Files\bundles\ebx"
outputFolder = r"M:\Research\FIFA17\Dump\EBX" ?.

It seems that ebx doesn't find anything.

Attachment:
ebx_console.jpg [278.47 KiB]
Not downloaded yet


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 12, 2016 9:47 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
@NoFaTe
What's the compression algorithm?
Is this "zstd"?
https://github.com/facebook/zstd

And do you know what's the algorithm (and relative source code) of the LZ77.dll which is floating around in older versions?


Top
   
 Post subject: Re: FIFA 17
PostPosted: Wed Oct 12, 2016 11:08 pm 

Joined: Tue Oct 11, 2016 5:10 pm
Posts: 4
aluigi wrote:
@NoFaTe
What's the compression algorithm?
Is this "zstd"?
https://github.com/facebook/zstd

And do you know what's the algorithm (and relative source code) of the LZ77.dll which is floating around in older versions?

Yes, the compression algorithm in use is a legacy version of zstd.
As for the LZ77 variant used in older versions, even though I don't have the source for the DLL itself, you might find these (#1 #2) interesting.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 28 posts ]  Go to page 1 2 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited