ZenHAX
https://zenhax.com/

Finding out compression out of custom image format
https://zenhax.com/viewtopic.php?f=7&t=6996
Page 1 of 1

Author:  metalbass [ Thu Nov 23, 2017 10:56 pm ]
Post subject:  Finding out compression out of custom image format

I'm currently extrating assets from Liberation Day (1998) in order to remake it.
I'm working on extracting .DBI files, which are archives of image assets.

I've managed to extract uncompressed images as 256bit indexed bitmaps from one of these DBI files.

It seems like there are some dbi files that are compressed, though. I'm trying to guess the compression without much luck. I'd need help or tips on how to find this.

Here you can find a link to the smallest (34KBs) DBI with compressed images file:
https://drive.google.com/file/d/1kPP3qP ... sp=sharing

You can jump to 0x434 and take the first 1133 bytes; that is the compressed image data.
I learned about this by decompiling the game code, but I cannot make any sense of the code that seems to decompress. I can upload snippets from that if it's helpful.

If you want to know more about the file structure, this is what I know from the header:
  • 0x0C Width: 54
  • 0x10 Height: 56
  • 0x14 Compressed: 1
  • 0x20 256-entry palette that takes 1024bytes

And what I know from the first image in the file:
  • 0x420 (+0x00) Name: MINE001
  • 0x428 (+0x08) Width: 54
  • 0x42A (+0x0A) Height: 56
  • 0x42C (+0x0C) Size of image color buffer in bytes: 1133
  • 0x434 (+0x20) Image color buffer.

One can find images at 0x420 + i * 0x2000.

Thanks!

Author:  BSM [ Fri Nov 24, 2017 6:54 am ]
Post subject:  Re: Finding out compression out of custom image format

At the end of the archive, at 0x00008420, you can find the directory, containing 4 entries:
"MINE000\0", Index 2
"GMINE000", Index 3
"MINE001\0", Index 0
"GMINE001", Index 1

Each of these inner files contains more than one single image, and other kind of data, like a list of terrains (index + name)
I don't know if you need all these data, so I will just show you how to decode pixels data.

At 0x00000434 (like you said), starts pixels data for the first image, and you can find 56 descriptors, one for each scan line of the image. After the descriptors, you will find a single byte with value = 0xCD, possibly used at End of Image as a check.
The structure of a line is:
Record Line
{
long lineDataLength
byte[lineDataLength - 4] lineData
}

If lineDataLength is greater than 8, you need to read pixels by repeating LinePixels records until all lineDataLength bytes are consumed

Record LinePixels
{
short length
short behavior
if (behavior == 0x0002)
byte[length] PixelsData
endif
}

An example could be handy:
08 00 00 00
lineDataLength = 0x00000008
so we read a LinePixels record:
36 00 01 00
length = 54 (the whole line for this image size)
behaviour = 0x0001 (transparent pixels)

Another example, this time with several chunks of data for a single line:
Code:
36 00 00 00    09 00 01 00    0C 00 02 00    DD 6F 8A ED 47 6C F7 57 34 5D ED 63 
            09 00 01 00    09 00 02 00    DD 5D 5D 6F B0 2F 61 37 39
            01 00 01 00    01 00 02 00    DD                               0D 00 01 00

lineDataLength = 54 (0x00000036)
read a chunk:
length = 9
behavior = 0x0001
so we add 9 transparent pixels and read the next chunk
length = 12
behavior = 0x0002
so we read 12 bytes, and write them to the output image, then read next chunk
length = 9
behavior = 0x0001
so we add another 9 transparent pixels and read the next chunk
length = 9
behavior = 0x0002
so we read 9 bytes, and write them to the output image, then read next chunk
length = 1
behavior = 0x0001
so we add another 1 transparent pixel and read the next chunk
length = 1
behavior = 0x0002
so we read 2 bytes, and write them to the output image, then read next chunk
length = 13
behavior = 0x0001
so we add another 13 transparent pixels and the line is completed, as we have read all the specified 54 bytes and the pixels count for this line is already the max

Some pseudo-code written here on the fly, not tested and not including errors check:
Code:
ReadLineOfPixels()
{
   int32 length = Data.ReadInt32();                  // Including this field
   int bytesRead = 4;
   while (bytesRead < length)
   {
      int16 amount = Data.ReadInt16();
      bytesRead += 2;
      int16 behavior = Data.ReadInt16();
      bytesRead += 2;
      if (behavior == 0x0001)
      {
         ImageOut.Write(amount, 0x00);             // Put "amount" of transparent pixels
      }
      elseif (behavior == 0x0002)
      {
         byte[] pixels = Data.ReadBytes(amount);          // Read "amount" of bytes, as indices in the palette
         bytesRead += amount;
         ImageOut.Write(pixels);
      }
      else
      {
         // Unsupported or Corrupted data; break program
      }
   }
}

Author:  metalbass [ Fri Nov 24, 2017 8:08 am ]
Post subject:  Re: Finding out compression out of custom image format

Thanks for the info and the quick reply.

I was specially blocked by this :)

After your explanation and pseudocode I understand this is a form of RLE encoding, right?
I'm trying to understand if this a named algorithm, so I can code this with it's proper naming :)

Thanks!

Author:  BSM [ Fri Nov 24, 2017 8:21 am ]
Post subject:  Re: Finding out compression out of custom image format

Yes, it is one of the multiple variations of RLE used in old games (the simplest one).

Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/