Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sun Sep 20, 2020 9:07 pm

All times are UTC

Post new topic  Reply to topic  [ 7 posts ] 
Author Message
PostPosted: Tue Oct 09, 2018 6:57 pm 

Joined: Tue Oct 09, 2018 5:42 pm
Posts: 4
A lot of games from Silmarils company use game files with .io extension and those are compressed. All these Silmarils games are full of .io files:

Crystals of Arborea
Metal Mutant
Ishar I
Ishar II
Ishar III
Transarctica (or Artic Baron)
Bunny Bricks

but there are more for sure, i have not checked them all. So discovering the compression algorithm will give us the possibility to decompress game data of a lot of Silmarils games.
The .io files store different kind of data: music, graphics, texts, etc.

Looks like a common file on all of those games is MAIN.IO, also BLANCPC.IO, sure there are others, i have not checked all.

In particular i have investigated Ishar: Legend of the fortress.

GAME DOWNLOAD (french version)
You can download the game from here (if you want to run tests and to investigate executing the game):

NOTE: Use the numeric pad of the keyboard to select the language when prompted.
NOTE2: The spanish version (not the one i posted up here) does not have the language selection screen! The file that displays that is MAIN.IO, so if you take the MAIN.IO from the french version and overwrite it on the spanish, you will get that screen.

All files are identical across spanish and original version except:

MAIN.IO (the one from french shows a language selection screen, spanish don't)
MESSAGE.IO (spanish version has this in spanish, and french version in french)
START.EXE (French version has some little differences like fading out the music on title screen when space bar is pressed, spanish version stops it abruptly)

I used WinMerge to compare across french and spanish version files:


The exe from Ishar is packed with PACK201 (PACK 2.01). I detected it using SH Archive Identifier (SHAID, Archid):

Then i checked it with PACK 2.01 itself and it detect that it is packed with it:

Them, i used "UUP v1.4 - Universal exe-file UnPacker" with the exe and successfully extracted it (and works if you execute it!):

The packed start.exe file i used for the screenshots are from the Spanish version.


Spanish unpacked START.EXE:
https://mega.nz/#!xdcwhI5L!jYOnoU1G6Ob9 ... BFHAuPpiyw
Packed EXE MD5:

French unpacked START.EXE:
https://mega.nz/#!IZEzhYbB!E-jmfGyfKyWz ... v3g0Ztvd78
Packed EXE MD5:

Size difference between unpacked EXE versions:
Spanish: 75 KB
French: 88 KB

A note when unpacking the French EXE version is that i get this message (that was not displayed on spanish EXE version) but the unpacked EXE works:

Now we can dissasembly the executable to do reverse engineering, unfortunately i don't know very much x86 assembler.

I run the game on Dosbox and i use the freeware tool ProcessActivityView over the Dosbox process to see what files the game access on every moment. You can also use ProcessMonitor that gives even more details: all the offsets on each file the game is accessing.



These are good to investigate because we know they have the same kind of data: two images of 320x200 (64.000 pixels), so we can compare each other and take a look at the header bytes to see differences, but unfortunately i have not found what seems to be the bytes that indicate the uncompressed data size.

- Starting the game, after some scenes (Silmarils logo and others) you will see a fullscreen image (320x200) of the game credits, the file used for this is AUTEUR.IO, it's like that:

Copy and pasting random little blocks of bytes from the file itself (AFTER what it looks like it is the header) over itself you can change the background of the image when the game displays it:

As you can see the background has been modified but not the red characters, but if you continue modifying blocks of bytes of the file, you will also modify the red characters (look on the bottom):

So looks like they are stored in two planes.

- When the player is dead, another fullscreen of 320x200 image will be shown:

OK. So now, the classic trick... What if we copy AUTEUR.IO file over DEAD.IO? Now supposedly, the credits must be shown when the player is dead but... you get a black screen (although music plays) and you can continue the game when pressing space. But I discovered how to get the image shown!

Comparing the first bytes of AUTEUR.IO and DEAD.IO you will see some little differences on the first bytes.

Looks like the byte on offset 16 (starting by 1 on the file) identifies something like the scene:
    AUTEUR.IO has 0x98 value
    DEAD.IO has 0xC0 value

So writing now the value of 0xC0 (the value that the dead scene file must have) over the 0x98 value to it works and now it displays the AUTEUR image when you're dead.

You can try my modified AUTEUR.IO file to see the dead screen instead of the credits screen (is a modified DEAD.IO file: it is the same except the byte at offset 16 that is taken from AUTEUR.IO to match the value that the game expects, because if it does not match, the image is not shown and you will see a black screen instead):

The strings are stored on those files. On the french version of the game:
    MESSAGE.IO: strings in french
    MESSAGED.IO: strings in german
    MESSAGEE.IO: strings in english
    MESSAGEI.IO: strings in italian

this is an advantage because they store the same strings but for different languages, so we can compare those files to examine how the bytes on the header are different because the file size differs, but i could not get where the uncompressed size bytes are on those MESSAGEX.io files :-(

On the french MESSAGE.IO file, i found 4 uncompressed strings:
  • "CARAC" (in characteristics panel in the game, "CARAC" is used to print the first part of the string "CARACTERISTIQUES :" and the first character "C" to print the first character of "CONSTITUTION : ")
  • "TU" (in characteristics panel in the game, "TU" is used to print the characters 7 and 8 in the string "CONSTITUTION : ")
  • "PHYS" (used to print "SCHLOUMZ : REGENERATION PHYSIQUE". This is a supposition, i have not checked this because you have to arrive to some part of the game to see that string. In ARAMIR warrior panel there is a string with "PHYSIQUE : " but it does not change if you change "PHYS" to another characters, so looks like it is used to print "SCHLOUMZ : REGENERATION PHYSIQUE")
  • "INVUL" (there are 2 strings that contain this string: "CLOPATOS : INVULNERABILITE" and "INVULNERABILITE 6" but i have not checked in the game if they change if i modify "INVUL")

You can change those strings with anothers characters, for example, you can change "CARAC" by "ABCDE" and you will see the changes in the game where they are used.

The "CARAC" and "TU" strings on MESSAGE.IO:

Used to print some word characters on characteristics panel:

Now, if you change "CARAC" and "TU" to "ABCDE" and "FG":

you will see the changes:

Also, "CARAC" string is used to print the competences panel:

so having "CARAC" changed to "ABCDE" you will see the changes:

Also, here are the "PHYS" and "INVUL" strings but i have not checked where they appear on the game:

So, the conclusion relative to compression is that MESSAGE.IO file:
  • has some uncompressed string parts: "CARAC", "TU", "PHYS" and "INVUL" (maybe there are more)
  • has the most string parts compressed (there are not so many kind of "valid" strings or dictionary words)
  • the most important, it use some kind of compression that characters are pointed from other strings to reuse them, as you can see in my examples: "CARAC" is the source string for the ingame strings "CARACTERISTIQUES", "ARMES 1 MAIN", "ARMES 2 MAINS", "ARMES DE JET" AND "COMPETENCES".

I have no compression algorithm knowledge but maybe some of you can help. As i told, discovering the Silmarils compression algorithm would make us having the possibility to decompress lot of game data from the dozens other games they have.

UNCOMPRESSED MESSAGE.IO (french and spanish)

I successfully extracted uncompressed MESSAGE.IO (french and spanish) from memory using the DOSBox debugger, you can download it here:

These files are what was written in memory when the file MESSAGE.IO was read, i carefuly executed it step by step with a breakpoint where the destination byte was written.

Anyone want to help?

Last edited by AleWin32 on Wed Oct 24, 2018 8:28 am, edited 31 times in total.

PostPosted: Tue Oct 09, 2018 8:19 pm 
Site Admin
User avatar

Joined: Wed Jul 30, 2014 9:32 pm
Posts: 11452
You forgot the most important thing: the files.

PostPosted: Tue Oct 09, 2018 8:41 pm 

Joined: Tue Oct 09, 2018 5:42 pm
Posts: 4
aluigi wrote:
You forgot the most important thing: the files.

Updated on my first post.

Last edited by AleWin32 on Wed Oct 10, 2018 10:06 am, edited 1 time in total.

PostPosted: Wed Oct 10, 2018 7:08 am 

Joined: Tue Oct 09, 2018 5:42 pm
Posts: 4
Game executable investigation added on my first post

PostPosted: Wed Oct 24, 2018 8:17 am 

Joined: Tue Oct 09, 2018 5:42 pm
Posts: 4
MESSAGE.IO file (french and spanish) uncompressed extracted from a memory dump on my first post, so now we have the compressed and the uncompressed file, but we don't know the compression algorithm...

PostPosted: Thu Jul 25, 2019 4:04 pm 

Joined: Wed Jul 24, 2019 5:57 pm
Posts: 1
While searching for a way to decompress Silmarils data files, I found this forum and want to share my findings.

It seems like the compression algorithm has already been found years ago. On eab there is a very old thread where extracting music files from Silmarils Amiga games is discussed. The interesting bit from this thread was a statement from a user named meynaf, who shared an unpacker (decruncher) for Silmarils games on the Amiga:

meynaf wrote:
Note that unlike WT's one, my xfd decruncher produces files that can still be used by the game engine.

I managed to find his file on the site’s ftp server and ran a few tests

I used the file accueil.*o (the main menu file) from the game Ishar 3. The versions I used are:

ACCUEIL.AO (25kb) - Atari ST german
ACCUEIL.CO (25kb) - Amiga OCS french
ACCUEIL.DO (75kb) - Amiga AGA multilingual (english, french, german)
ACCUEIL.FO (75kb) - Macintosh / Atari Falcon multilingual (english, french, german)
ACCUEIL.IO (75kb) - DOS spanish

All files are compressed. The size difference between the Atari ST/ Amiga OCS versions and the rest is simply the lower color palette and resolution of the first two versions. The file ACCUEIL contains the texts and the graphics of the main menu.

Atari ST
Amiga AGA

Testing Environment

First of all, let me share my testing environment, so you can play with it yourself.:


In the Amiga Folder there is a batch file named workbench.bat which will start FS-UAE with a preinstalled Workbench environment. The folder .\Hard Drives\Playground is mounted as HD, so we can easily transfer files from or to the Amiga, The decruncher from the eab thread is already installed in workbench.

The second batch file in the amiga folder is named Ishar3.bat and will boot the game directly from the Folder .\Hard Drives\Ishar3

But for now, let’s start with the workbench.
In the folder Project Silmarils\Data\Multiformat Comparison\Untouched Files I have collected some data files from different Ishar 3 platform releases. We copy all files to our Playground folder and boot up the workbench.

Test #1 - accueil.do

Let’s start with accueil.do
In a shell window we type
Xfddecrunch accueil.do accueil2.do


The extraction is successful and we can leave the Amiga for now (Alt+F4)
The new accueil2.do has a size of 129kb.
We open it in a hex editor

Et voila. The texts of the main menu. But will the game run with the uncompressed file? Let’s find out.
We rename the file back to accueil.do and copy it to /Hard Drives/Ishar3
Now we start ishar3.bat and…

The game works :D
So it safe to assume that Silmarils games can indeed run no matter if the data is compressed or extracted. At least on the Amiga.
But what about the other platforms?

Test #2 - accueil.ao, .co & .fo

We now try to extract the Amiga OCS, Atari St & Macintosh/falcon files.
All files are recognized by the decruncher and are extracted without problems.

These are extracted create.*o (character creation menu) files, but accueil works as well

I havent tested extracted .ao & .fo yet, so I don't know, if the game will run.
I have tested the Amiga OCS/AGA files and they work fine.

Test #3 - accueil.io DOS

Now for the DOS files (.io)
The decruncher fails to recognize them as compressed so we cannot extract them. Or can we?


If we modify the dos files by copying the first few bytes of the Amiga AGA files, the decruncher recognizes and extracts them. By opening them in a hex editor we can confirm, that the extraction was successful.


But will the file work?
We can rename and copy it to the DOS/Ishar3 folder and start dosbox.
The game crashes with an error code.

So I assume that the data itself was extracted correctly, but the file informations/headers are incorrect. Or maybe the DOS version doesn’t work with uncompressed files.

Test #4 - Does the DOS version work with uncompressed files?

And now for the reason I used the spanish version of Ishar3 in Dos.
The floppy version comes with a compressed accueil.io file (75kb), but we have access to an uncompressed version.
The Amiga AGA version has 3 different languages: english, french and german. The file accueil.do contains the texts in all 3 languages.
But the DOS CD-Rom release added a 4th language: spanish. The original accueil.io still only contains the 3 languages, but a second file named accueils.io contains the spanish texts only. And by looking at the file size (109kb) and in a hex editor we can see that this file is uncompressed.

I have added both files to /Data/Ishar 3 Dos and renamed accueils.io to accueil.io
If we copy the uncompressed accueil.io to DOS/Ishar3 and start the game, it will boot just fine.


We have an extraction tool that will give us uncompressed, usable files for the Amiga version of Silmarils games.
Atari and Macintosh extraction seems to work but resulting files are untested for now.
PC extraction only works with modified files, but uncompressed files are unusable. Our uncompressed file has a size of 129kb, the original uncompressed file that works with the game has a size of 109kb
We have an original compressed and an uncompressed accueil.io that can be compared.

And the most important thing: The amiga decruncher library comes with the source code, so someone with programming skills should be able to use it for further examination.
I have absolutely no programming knowledge, so i can't do anything with the source code.

; silmarils file decruncher, basé sur le mien et celui de wanted team

marge equ 256         ; si la décompression déborde du buffer

 moveq #-1,d0
 dc.b "XFDF"         ; xfd id
 dc.w 1            ; version
 dc.w 0            ; reserved
 dc.l 0,0         ; private
 dc.l slave1         ; first slave

 dc.b "$VER: XFD Silmarils file decruncher 1.01 (17.11.2002)",0

name1 dc.b "Old Silmarils",0
name2 dc.b "Old Silmarils Main",0
name3 dc.b "Silmarils",0
name4 dc.b "Silmarils Main",0
name5 dc.b "Old Silmarils (interlaced)",0
name6 dc.b "Old Silmarils Main (interlaced)",0

; macro declare slave : n°, next, minsize
declslv macro
 ifne \2
 dc.l slave\2      ; next
 dc.l 0         ; dernier
 dc.w 2         ; version
 dc.w 39      ; xfdmaster version
 dc.l name\1      ; format name
 dc.w $304      ; flags
 dc.w 0         ; maxlen special info
 dc.l check\1      ; check routine
 dc.l decr\1      ; decrunch routine
 dc.l 0         ; recog segment
 dc.l 0         ; decrunch segment
 dc.w 0,0      ; slave/replace id
 dc.l \3      ; min size (wanted team met 14)

 declslv 1,2,8
 declslv 2,3,$18
 declslv 3,4,$10
 declslv 4,5,$20
 declslv 5,6,8
 declslv 6,0,$18

; note : bug dans les premiers xfd silmarils decrunchers : NetBSD executables
; -> je peux pas tester...

; check (main avec b0=0 car 4(a0) sera aussi 0, donc c'est égal)
check1 moveq #1,d1         ; old
 bra.s check
check2 moveq #0,d1         ; old main
 bra.s check
check3 moveq #3,d1         ; new
 bra.s check
check4 moveq #2,d1         ; new main
 bra.s check
check5 moveq #5,d1         ; old (interlace)
 bra.s check
check6 moveq #4,d1         ; old main (interlace)
 sub.l (a0),d0            ; d0=taille24bits : crunch-norm
 btst #23,d0            ; signe du résultat
 beq.s .oups            ; taille crunchée > taille normale ?!
 move.b (a0),d0
 andi.b #$9f,d0            ; 81(old) ou a1(new)
 cmpi.b #$81,d0            ; (wt accepte aussi 80/C0)
 bne.s .oups
 move.w 4(a0),d0         ; 0000 main.co 0001 data.co
 eor.b d0,d1            ; bits 0 identiques -> b0 d1 à 0
 andi.w #$fffe,d0         ; le reste doit être 0
 bne.s .oups
 move.b (a0),d0            ; bit 5 nous intéresse (old/new)
 andi.b #$20,d0
 lsr.b #4,d0            ; -> en bit 1
 eor.b d0,d1            ; bits 1 identiques -> b1 d1 à 0
 move.b (a0),d0
 andi.b #$40,d0            ; bit 6, cette fois
 lsr.b #4,d0            ; -> en bit 2
 eor.b d0,d1
 bne.s .oups            ; (donc d1 totalement à 0)
 move.l (a0),d0
 andi.l #$ffffff,d0
 cmpi.l #$100000,d0         ; jamais >1m (512k) dans jeux silmarils
 ble.s .ok            ; (record : montagne.do, 241k-ishar3)
.oups moveq #0,d0         ; pas bon
 move.l d0,4(a1)         ; taille saved data
 add.l #marge,d0
 move.l d0,(a1)
 moveq #1,d0            ; indiquer ok

; decrunch routines
decr1 moveq #0,d1
 bra.s decr
decr2 moveq #1,d1
 bra.s decr
decr3 moveq #2,d1
 bra.s decr
decr4 moveq #3,d1
 bra.s decr
decr5 moveq #4,d1
 bra.s decr
decr6 moveq #5,d1
 bra.s decr
 movem.l d2-d7/a2-a6,-(a7)   ; si lui le fait, y a une raison
 move.l $3c(a0),a1      ; outbuf
 move.l a1,a2
 move.l 4(a0),d6      ; src size
 move.l (a0),a0         ; src addr
 move.l (a0),d7
 andi.l #$ffffff,d7      ; dest size
 addq.l #1,a0
 moveq #4,d0         ; norm header (6 bytes)
 btst #0,d1
 beq.s .norm
 moveq #$14,d0         ; main header ($16 bytes)
.norm move.b #1,(a1)+
.loop move.b (a0)+,(a1)+
 dbf d0,.loop
 add.l a2,d7         ; -> end of dest buff
 sub.l a1,d7         ; -> reste de dest à remplir
 btst #1,d1
 beq.s .old
 bsr newm         ; new method (>=ishar1)
 bra.s .done
 bsr.s oldm         ; old method (<ishar1)
 movem.l (a7)+,d2-d7/a2-a6
 moveq #1,d0

; decrunch routines : reçoivent a0/d6=src, a1/d7=dest (adr/taille)
; si erreur : compléter avec octets nuls (ça se produit sur certains fichiers)
; si output buffer dépassé, quitter simplement

; ancienne méthode
 lea (a0,d6.l),a2
 lea (a1,d7.l),a3
 moveq #1,d3
 btst #2,d1
 beq.s .hmpf
 moveq #8,d3
.hmpf move.l d3,d4
.loop0 cmpa.l a1,a3
 ble.s .fin
 bsr.s .in
 move.b d1,d2
 bclr #7,d2
 bne.s .rep
.loop1 bsr.s .in
 bsr.s .out
 subq.b #1,d2
 bne.s .loop1
 bra.s .loop0
.rep bsr.s .in
.loop2 bsr.s .out
 subq.b #1,d2
 bne.s .loop2
 bra.s .loop0
.in cmpa.l a0,a2
 blt.s .zero
 move.b (a0)+,d1
.zero moveq #0,d1
.out move.b d1,(a1)
 add.l d3,a1
.fin sub.l d7,a1
 addq.l #1,a1
 subq.l #1,d4
 bne.s .loop0

; nouvelle méthode
 lea (a0,d6.l),a4      ; end of input buffer
 lea (a1,d7.l),a2      ; end of output buffer
; code identique à partir de là
 subq.l #8,a7
 move.l (a0)+,(a7)
 move.l (a0)+,4(a7)
 clr.w d7
.loop cmpa.l a2,a1
 bgt.s .exit
 moveq #1,d0
 bsr.s .vache
 tst.b d5
 beq.s .momie
 moveq #0,d2
.zombie moveq #2,d0
 bsr.s .vache
 add.w d5,d2
 cmp.w #3,d5
 beq.s .zombie
.qqch moveq #8,d0
 bsr.s .vache
 move.b d5,(a1)+
 dbf d2,.qqch
 cmpa.l a2,a1
 bgt.s .exit
.momie bsr.s .bouh
 clr.w d0
 move.b 0(a7,d5.w),d0
 andi.w #3,d5
 beq .seins
 move.w d5,d2
 bsr.s .vache
.femelle neg.w d5
.sexe move.b -1(a1,d5.w),(a1)+
 dbf d2,.sexe
 bra.s .loop
.seins bsr .vache
 move.w d5,d3
 clr.w d2
.cervelle bsr.s .bouh
 add.w d5,d2
 cmp.w #7,d5
 beq.s .cervelle
 move.w d3,d5
 addq.w #4,d2
 bra.s .femelle
.bouh moveq #3,d0
.vache sub.b d0,d7
 bmi.s .worm
 clr.w d5
 rol.l d0,d5
.worm add.b d0,d7
 clr.w d5
 rol.l d7,d5
 swap d5
 cmpa.l a4,a0      ; ces deux lignes sont un ajout à moi
 bge.s .argh      ;
 move.w (a0)+,d5
 swap d5
 sub.b d7,d0
 moveq #$10,d7
 rol.l d0,d5
 sub.b d0,d7
; tout ce qui suit est ajouté par moi
 addq.l #4,a7
.zero clr.b (a1)+
 cmpa.l a1,a2
 bhi.s .zero
 addq.l #8,a7

PostPosted: Sun Sep 13, 2020 9:58 pm 

Joined: Sun Sep 13, 2020 9:38 pm
Posts: 1
For DOS version

Bytes 00-01 ~ H0
Bytes 02-03 ~ H1
Bytes 04-05 ~ H2 - ExtraHeader(Read next 16 bytes)
HeaderSize = 6 or 22(6 basic and 16 extra)
Unpacked size = SWAP16(H1&0xFF)*16+(H0-HeaderSize)
First logic switch (upper bit of H1), 0-uncompressed 1-compressed
If file compressed next logic switch is upper 7 bits of H1, program compare it with 0xA0 (maybe just check bit 13 of H1, because bit 15 checked on previous step and =1)

Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 7 posts ] 

All times are UTC

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited