ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Mon Oct 23, 2017 2:39 am

All times are UTC




Post new topic  Reply to topic  [ 22 posts ]  Go to page 1 2 Next
Author Message
PostPosted: Tue Oct 03, 2017 5:22 pm 

Joined: Fri Oct 09, 2015 3:38 pm
Posts: 26
Hi, previous FIFA 17 was known to contain ZSTD streams and it was pretty easy to pick them up by their headers. FIFA 18 also seems to have ZSTD streams but at many places the headers are not even complete (modified?) so its hard to pick ZSTD streams and dump them out.

Can anyone please find a solution to dump all the found ZSTD streams in a CAS file of this game ?

Sample file:
Code:
https://www.mediafire.com/file/f5ovyzca66532ys/cas_01.cas

If you want a smaller sample please tell here.


Top
   
PostPosted: Tue Oct 03, 2017 6:37 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
Can you provide a sample of FIFA17 (maybe the same cas_01.cas if available) so we can check what changed?


Top
   
PostPosted: Tue Oct 03, 2017 7:30 pm 

Joined: Fri Oct 09, 2015 3:38 pm
Posts: 26
aluigi wrote:
Can you provide a sample of FIFA17 (maybe the same cas_01.cas if available) so we can check what changed?


Here:
Code:
http://www4.zippyshare.com/v/mwoIobM3/file.html


Top
   
PostPosted: Tue Oct 03, 2017 9:51 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
Ok so the format is the one covered by frostbite.bms.
Do you have the "cat" of fifa18?


Top
   
PostPosted: Wed Oct 04, 2017 1:48 pm 

Joined: Fri Oct 09, 2015 3:38 pm
Posts: 26
aluigi wrote:
Ok so the format is the one covered by frostbite.bms.
Do you have the "cat" of fifa18?


Ok, I have attached the cas.cat file associated with that cas_01.cas.
Also here's another set of FIFA 18 samples in case you need it:
Code:
http://www91.zippyshare.com/v/jC5IBBre/file.html


Attachments:
cas.zip [2.17 KiB]
Downloaded 9 times
Top
   
PostPosted: Thu Oct 05, 2017 9:59 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
I bet the problem is the finalSize 32bit field that apparently is a 24bit:
Code:
01 00 04 60 0f 70 00 b0
So 0x460 is read as 0x01000460.

Are you using FB2Dumper.py?
If yes then find the following line:
Code:
finalSize, magic, payloadLen = unpack(">IHH", cas.read(8))
and add this line after it:
Code:
finalSize &= 0x00ffffff


Top
   
PostPosted: Thu Oct 05, 2017 10:44 am 

Joined: Fri Oct 09, 2015 3:38 pm
Posts: 26
aluigi wrote:
Are you using FB2Dumper.py?


Actually I only want to dump out zstd compressed streams, both in compressed (raw,as-it-is) and decompressed forms. Can you help with it ?


Top
   
PostPosted: Thu Oct 05, 2017 12:17 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
No, I can't.
First because it's not clear what type of zstd it's used in fifa18 since zstd06 gives just invalid files and then some chunks use oodle or are no compressed at all.

What I did was updating my frostbite script to version 0.3.1, I will upload it in the next hours


Top
   
PostPosted: Thu Oct 05, 2017 1:06 pm 

Joined: Fri Oct 09, 2015 3:38 pm
Posts: 26
aluigi wrote:
No, I can't.
First because it's not clear what type of zstd it's used in fifa18 since zstd06 gives just invalid files and then some chunks use oodle or are no compressed at all.

What I did was updating my frostbite script to version 0.3.1, I will upload it in the next hours

Hmmm maybe thats why in most of the cas files I wasnt even able to detect zstd unlike in FIFA 17 where I could find zstd everywhere, in FIFA 18 along with very few legit zstd chunks, only part of zstd headers were there so maybe they are not even zstd, they were oodle instead like you said.


Top
   
PostPosted: Thu Oct 05, 2017 1:49 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
What's strange is that these invalid zstd chunks have the same 0x0f70 flag of the good ones, the only difference is that 0x01000000 added to the decompressed chunk size which is just another flag (so the size is 24bit).
Trying all the legacy zstd gave the same invalid results and even scanning all the compression algorithms was the same.


Top
   
PostPosted: Thu Oct 05, 2017 10:36 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
The new script is available:
http://aluigi.org/bms/frostbite.bms


Top
   
PostPosted: Sat Oct 07, 2017 2:58 pm 

Joined: Fri Aug 08, 2014 6:24 am
Posts: 11
ZSTD files in FIFA 18 are using pre defined dictiomary.
The buffer size has changed from 64K to 256K but the compressed size in header is now 24 bits and because of that we see 0x70 , 0x71,0x72,0x73.
The lower bits should be added to compressed size.

And the pre defined ditionary is encrypted in initfs_Win32.


Top
   
PostPosted: Sat Oct 07, 2017 5:37 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
Interesting.
So first of all we need the dictionary to use when the first 8bit flag is 1.
Anyone? :)

That thing regarding 0x7* is not very clear, for example here I have "00 04 00 00 00 74 00 00" so I must copy 0x40000 bytes without compression to destination and should I append 4 zeroes to it?

In the meantime there was a typo in my script (FLAGS >= 0 instead of != 0) so I have updated it.


Top
   
PostPosted: Mon Oct 09, 2017 9:17 am 

Joined: Fri Aug 08, 2014 6:24 am
Posts: 11
aluigi wrote:

That thing regarding 0x7* is not very clear, for example here I have "00 04 00 00 00 74 00 00" so I must copy 0x40000 bytes without compression to destination and should I append 4 zeroes to it?

you should copy 0x40000 bytes without compression and you don't need to append anything.
I have decrypted initfs and obtained the dictionary and I have tested it.
Decompression works fine.

If there's no legal concerns I can share it.


Top
   
PostPosted: Mon Oct 09, 2017 9:22 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
So why you said "The lower bits should be added to compressed size"?
That means 0x40004 based on what you said.

Yeah, share the dict :)


Top
   
PostPosted: Mon Oct 09, 2017 11:11 am 

Joined: Fri Aug 08, 2014 6:24 am
Posts: 11
aluigi wrote:
So why you said "The lower bits should be added to compressed size"?


What I meant was :
00 04 00 00 15 71 A9 9C => ((71 & 0xF) << 16 ) | 0xA99C => 0x1A99C

Here's the Dictionary. :twisted: :twisted:


Attachments:
dic.rar [60.85 KiB]
Downloaded 3 times
Top
   
PostPosted: Mon Oct 09, 2017 11:55 am 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
Does the game still use zstd 0.6?
Because I still get invalid data using the dictionary


Top
   
PostPosted: Mon Oct 09, 2017 12:06 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
I answer to myself: no, it uses the new one.
The bad thing is that there is no way to know if it's the old (fifa17) or new one (fifa18)


Top
   
PostPosted: Mon Oct 09, 2017 12:16 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 6904
Script 0.3.3 :D
http://aluigi.org/bms/frostbite.bms


Top
   
PostPosted: Mon Oct 09, 2017 12:21 pm 

Joined: Fri Aug 08, 2014 6:24 am
Posts: 11
aluigi wrote:
The bad thing is that there is no way to know if it's the old (fifa17) or new one (fifa18)


Actually there is.
Fifa17 zstd compression starts with : 0xFD2FB526
Fifa18 zstd compression starts with : 0xFD2FB528

Of course if you mean the compression.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 22 posts ]  Go to page 1 2 Next

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited