ZenHAX
https://zenhax.com/

How to recognize the compression algorithms with your eyes
https://zenhax.com/viewtopic.php?f=4&t=27
Page 2 of 2

Author:  Vido [ Sun Jan 14, 2018 4:18 am ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

Firmwares for SOHO routers are using LZMA to pack firmwares, there is tool called binwalk, its written in python and if you using python 2 and have python-lzma installed binwalik will validate found headers an false-positives will be excluded from search, something simmilar could be done for quickbms I think, its idea

Author:  DavidDineen [ Thu Aug 16, 2018 7:27 am ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

aluigi wrote:
Good. Anyway I have added view view the phen375 results zstd to the list in the first post since it uses a magic number that allow to guess it at 100%.

Was looking for zstd compression recognising technique and was about to ask in a thread. But here I found 100% success technique. Thanks for mentioning the magic number, this solves my problem I was facing in my current project.

Author:  aluigi [ Wed Dec 12, 2018 10:00 pm ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

In case someone is interested in some statistics, I have collected the most used compression algorithms used in my 2'023 quickbms scripts.
Some games uses zip-like archives and that means some numbers may be inflated, some scripts cover multiple formats and some algorithms may be invoked twice in few scripts just because they are for more than one format.
Code:
zlib    545
deflate 119
lzma    87
xmemdecompress (lzx) 61
lzss    56
lzo1x   49
gzip    49
lz4     44
lz77wii 21
oodle   21
bzip2   16
dk2/EA  15
zstd    10
custom  10

Author:  GHFear [ Sat Mar 23, 2019 12:52 am ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

Thanks! This has been very helpful lately :)
Just want to add how I spot RefPack/dk2 nowadays.
First off...
0xXX = random value.
RefPack/dk2 compression mostly looks like this "0xXX\0xFB" (where the 0xFB seems to be the tell tale sign of refpack.) and then right after is the Decompressed Size as a 4byte 32 bit integer / long. (to further confirm that it in fact is refpack/dk2.)
So all in all it looks like this "0xXX\0xFB\0x00\0x1B\0x62\0x1C"

Depending on the strength of the compression the strings can still be sorta readable like this: "c:\datatemp\inter\PS2\Neutral\Chapâs\Hogwarts\ZoneãONE_HW_Viaduct_E.nãnce_DD\LefTextur�'àET_BURNâOOK_PAPER.ss€"

If you see this pattern, use comtype dk2

Author:  baloons34 [ Tue Jun 11, 2019 1:15 pm ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

GHFear wrote:
Thanks! This has been very helpful lately :)
Just want to add how I spot RefPack/dk2 nowadays.
First off...
0xXX = random value.
RefPack/dk2 compression mostly looks like this "0xXX\0xFB" (where the 0xFB seems to be the tell tale sign of refpack.) and then avis sur phenq france right after is the Decompressed Size as a 4byte 32 bit integer / long. (to further confirm that it in fact is refpack/dk2.)
So all in all it looks like this "0xXX\0xFB\0x00\0x1B\0x62\0x1C"

Depending on the strength of the compression the strings can still be sorta readable like this: "c:\datatemp\inter\PS2\Neutral\Chapâs\Hogwarts\ZoneãONE_HW_Viaduct_E.nãnce_DD\LefTextur�'àET_BURNâOOK_PAPER.ss€"

If you see this pattern, use comtype dk2

Thank you so much, all this info is so useful. I have spent hours on Google trying to get things straight for my new project and then found this thread which is just full of information. The zstd code is exactly what I was not getting right. You saved my time!

Author:  GHFear [ Tue Jun 11, 2019 2:16 pm ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

baloons34 wrote:
Thank you so much, all this info is so useful. I have spent hours on Google trying to get things straight for my new project and then found this thread which is just full of information. The zstd code is exactly what I was not getting right. You saved my time!


I think you meant to thank aluigi :)

aluigi wrote:
This Guy!

Author:  aluigi [ Wed Jun 12, 2019 8:42 am ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

:D

Author:  baloons34 [ Sun Jun 23, 2019 11:55 am ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

GHFear wrote:
baloons34 wrote:
Thank you so much, all this info is so useful. I have spent hours on Google trying to get things straight for my new project and then found this thread which is just full of information. The zstd code is exactly what I was not getting right. You saved my time!


I think you meant to thank aluigi :)

aluigi wrote:
This Guy!


Hahahaha yes you are right, I must thank alugi :D

Author:  baloons34 [ Sun Jun 23, 2019 11:56 am ]
Post subject:  Re: How to recognize the compression algorithms with your eyes

aluigi wrote:
:D


Thanks a lot Alugi :)

Page 2 of 2 All times are UTC
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/