ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Mon Sep 24, 2018 3:19 am

All times are UTC




Post new topic  Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Surviving Mars .hpk
PostPosted: Tue Mar 20, 2018 4:29 pm 

Joined: Sun Jan 18, 2015 11:22 pm
Posts: 34
hpk format:
Code:
// header, 36 bytes
char magick[4]; // "BPUL"
uint unk1;      // 36 (header size or data offset??)
uint unk2[5];   // 1, -1, 0, 0, 1

uint entry_offset;  // absolute
uint entry_size;    // entry_offset + entry_size = EOF

// data
char filedata[xxx]; // FOURCC can be "LZ4 " or other

// file desc
struct dir_entry[xxx] {
    uint ptr;   // entry[ptr] -> next item
    uint attr;  // 0 - directory, 1 - file
    short sz;
    char name[sz]   // name of current dir or file
}

// entry_offset
struct entry[xxx] {   // xxx = entry_size >> 3
    uint offset;        // absolute, pointer to dir_entry (included dir, file or filedata)
    uint size;          // size of dir_entry or zsize of file
}


LZ4 packed file:
Code:
char magik[4];          // "LZ4\x20"
uint size;              // unpacked size
uint block_size;        // 2^17
uint h_size;            // header size; if size==0, then EOF here
uint chunk_offset[xxx]; // xxx = (h_size - 16) >> 2, absolute offsets of chunk begins

struct chunk[xxx] {     // offset = h_size
    char lz_data[];     // full LZ4 block (unpacked == block_size)
}
char lz_data_tail[];    // partial LZ4 block (unpacked < block_size)


for unpack can be used LZ4_decompress_safe (src, dst, zsize, size). if (entry.size == size - h_size) then file content is not packed.

implementation in Lua here.


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Tue Mar 20, 2018 8:12 pm 

Joined: Wed Aug 13, 2014 10:00 am
Posts: 147
Or you can use HPK Archiver, modded by Atom0s

https://github.com/atom0s/HpkArchiver

Win build:
Attachment:
HPK_archiver_1_0_14_mod.7z [175.74 KiB]
Downloaded 199 times


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Tue Mar 20, 2018 9:10 pm 

Joined: Sun Jan 18, 2015 11:22 pm
Posts: 34
"quick&dirty" BMS-script for unpack lz4 compressed files (I hope that someone optimizes this code ;)
Code:
idstring "LZ4 "
comtype lz4

#### bad example, output is corrupt
get SIZE long
get BLOCK long
get H_SIZE long
get ZSIZE ASIZE
math ZSIZE -= H_SIZE

clog "milky_way_bad.dds" H_SIZE ZSIZE SIZE


goto 4 #### back to begin

#### good example, CRC32 1AA851A4
get SIZE long
get BLOCK long
get H_SIZE long
get ZSIZE ASIZE
xmath CHUNKS "(H_SIZE - 16) / 4"

math I = 0
set CHUNK_OFF[I] H_SIZE
for I = 1 <= CHUNKS
    get CHUNK_OFF[I] long
next I
set CHUNK_OFF[I] ZSIZE

math CHUNKS + 1
for I = 1 <= CHUNKS
    math VAR = CHUNK_OFF[I]
    math I - 1
    math VAR - CHUNK_OFF[I]
    set CHUNK_SZ[I] = VAR
next I + 2
math CHUNKS - 1

for I = 0 < CHUNKS
    clog "milky_way_good.dds" CHUNK_OFF[I] CHUNK_SZ[I] BLOCK
    if I == 0
        append
    endif
next i
xmath TAIL "SIZE - CHUNKS * BLOCK"
clog "milky_way_good.dds" CHUNK_OFF[I] CHUNK_SZ[I] TAIL
append


Attachments:
File comment: example of LZ4 compressed data
milky_way.zip [601.72 KiB]
Downloaded 45 times
Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Tue Mar 27, 2018 9:49 pm 

Joined: Tue Mar 27, 2018 9:48 pm
Posts: 4
Why does the script not save the file under the file's input name?
You can not do it? :roll: :lol:

rm_mist.tga
+ 0006fada 18 milky_way_good.dds
Info: algorithm 249
offset 0006fada
input size 0x00000012 18
output size 0x00000012 18
result 0xfffffffc -4

Error: the uncompressed data (-4) is bigger than the allocated buffer (131072)

=============================================
Most of the "tga" files work.
In "lua" it only cuts out a few bytes from the beginning of the file.


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Wed Mar 28, 2018 8:05 am 

Joined: Sun Jan 18, 2015 11:22 pm
Posts: 34
zbychos wrote:
Why does the script not save the file under the file's input name?

because this is a test case for a single file. in your case, the input size is equal to the output, that is, no compression is used.

Quote:
In "lua" it only cuts out a few bytes from the beginning of the file.

can an example? the name of the hpk-archive and the "cutted" file?


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Wed Mar 28, 2018 9:38 am 
User avatar

Joined: Sun Aug 17, 2014 7:27 pm
Posts: 1143
I found decompressor
Code:
idstring "LZ4 "
comtype lz4
get SIZE long
get CHUNK_SIZE long
get NAME filename
xmath TMP "SIZE % CHUNK_SIZE"
if TMP != 0
   math TMP = 1
else
   math TMP = 0
endif
xmath CHUNKS "(SIZE / CHUNK_SIZE) + TMP"
append
for i = 1 <= CHUNKS
   get OFFSET long
   savepos POS
   if i == CHUNKS
      get ZSIZE asize
   else
      get ZSIZE long
   endif
   math ZSIZE -= OFFSET
   goto POS
   if ZSIZE < CHUNK_SIZE
      clog NAME OFFSET ZSIZE CHUNK_SIZE
   else
      log NAME OFFSET ZSIZE
   endif
next i
append


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Wed Mar 28, 2018 6:07 pm 

Joined: Tue Mar 27, 2018 9:48 pm
Posts: 4
hhrhhr wrote:
zbychos wrote:
Why does the script not save the file under the file's input name?

because this is a test case for a single file. in your case, the input size is equal to the output, that is, no compression is used.

Quote:
In "lua" it only cuts out a few bytes from the beginning of the file.

can an example? the name of the hpk-archive and the "cutted" file?


hhrhhr
I apologize for my joke but I could not resist ...: D.
I know it's just an example.
OK, the end of jokes.

I used HPK Archiver

graphics catalog - UI
file UI\Icons\Research\rm_mist.tga
It may be a bug in the file but I have hpk once again and it's the same.

Lua 'Data' catalog
It does not work with any * .lua file
see lua before.rar and lua after.rar

Such a request: can you give a way to store the directory structure in the 'output' directory?

get FNAME filename
get FFOLDER input_folder
string NAME += FFOLDER
string NAME += "/"
string NAME += FNAME

It works more or less, but not as I expected. I have never had contact with QuickBMS before and for now I find it difficult to understand the basics.


Attachments:
Lua AFTER.rar [4.87 KiB]
Downloaded 29 times
Lua BEFORE.rar [6.08 KiB]
Downloaded 28 times


Last edited by zbychos on Wed Mar 28, 2018 7:36 pm, edited 4 times in total.
Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Wed Mar 28, 2018 6:12 pm 

Joined: Tue Mar 27, 2018 9:48 pm
Posts: 4
makc_ar wrote:
I found decompressor


makc_ar
I do not want to worry you, but it's cleansed and improved the same script.
In action nothing changed :)


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Wed Mar 28, 2018 6:25 pm 
User avatar

Joined: Sun Aug 17, 2014 7:27 pm
Posts: 1143
zbychos
Try this viewtopic.php?f=9&t=2630


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Wed Mar 28, 2018 10:41 pm 

Joined: Sun Jan 18, 2015 11:22 pm
Posts: 34
zbychos wrote:
I used HPK Archiver

I'm not the author of this program.

Quote:
Such a request: can you give a way to store the directory structure in the 'output' directory?

if you have in mind the finalization of the bms-script, then I'm not interested. for full unpacking there is the aforementioned HPK Archiver. my lua script on githab (hpk_unpack.lua) also unpacks all lz4 compressed files.


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Fri Mar 30, 2018 1:22 am 

Joined: Tue Mar 27, 2018 9:48 pm
Posts: 4
OK no problem. Thanx


Top
   
 Post subject: Re: Surviving Mars .hpk
PostPosted: Fri Apr 27, 2018 12:34 pm 

Joined: Fri Apr 27, 2018 12:27 pm
Posts: 1
Someone in the world have figured out how to convert the .hgm file?


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 12 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited