About Compiling QuickBMS In Windows
Page 1 of 1

Author:  Brolijah [ Wed Jun 05, 2019 10:57 pm ]
Post subject:  About Compiling QuickBMS In Windows

I finally got this to work under Windows, but because of the amount of time I spent trying to make this work, I am compelled to share my experiences and comments on doing so. I don't want others to waste nearly as much time as I did trying to make this work.

I was initially unsure whether to make this thread under Tutorials or here. I settled for here because I wanted to also make some comments on the build scripts and compatibility. Prior to simply asking in the QBMS Errors thread if there was a Windows specific build script, I could find no mention of there being one on aluigi.altervista.org nor in this forum. Nor could I find any mention of a user having attempted to compile under Windows.

I'll spare most the experience of wandering in the dark trying to build using the makefile prior, but much of the changes I had to make with the makefile were present in that batch script already. For future reference to curious individuals: Don't try to build under Windows with the Makefile. It's hell and you'll waste more time chasing dubious compilation errors than you may want to admit.

For convenience, I'll quote aluigi's build script here:
aluigi wrote:
I use a .bat file for doing the whole compiling job on Windows using gcc 4:

I also made some minor edits to the script for convenience, readability, and a couple comments about compilation:
File comment: Slightly modified batch script to build quickbms in windows.
compile_quickbms.zip [2.51 KiB]
Downloaded 19 times

Some of the following might be common sense solutions for some readers, but I'd rather be thorough and complete than not.

Potential Pitfalls
1. Missing includes to library headers shipped with MinGW.
src\quickbms.c:172:14: fatal error: openssl/crypto.h: No such file or directory
     #include <openssl/crypto.h>
For some reason this occurred with all my MinGW installations. Headers and libs located in the opt directory aren't in the search paths by default (ie OpenSSL). As such, you may want to manually add opt\include and opt\lib paths to the compilation script.

2. Missing includes for mcrypt and tomcrypt.
Add these include paths to the compilation script. (My modified script already has this addition.)
-Isrc\libs\libmcrypt -Isrc\libs\libtomcrypt

3. Several "skipping incompatible" library errors while linking.
You're trying to use an x86_64 toolchain + libraries. Install the i686 toolchain and try with that.

4. Failure to link xcompress.lib
ld.exe: xcompress.lib(obj/i386/lci.obj): warning: line number count (0x2) exceeds section size (0x1)
xcompress.lib(obj/i386/lci.obj): could not read symbols: Invalid operation
collect2.exe: error: ld returned 1 exit status
Don't use GCC 7 or newer. The library xcompress.lib will be incompatible with the binutils that are shipped with GNU GCC 7 and 8. If you're hellbent on using latest possible GCC toolchain, then use GCC 6.4.0. It will also work with GCC 5 and 4.

5. Undefined reference to `BCrypt_init`
It's possible that bcrypt.h has been included by another file (possibly wincrypt.h). I've observed this in both GCC 5 and GCC 6. So, unless you're Mr. Aluigi himself or you actually are attempting to build and test on a Win98 machine... It should be satisfactory enough to make these two changes and add -lbcrypt to the compilation script:
// In file /src/myenc.c : Comment out the mybcrypt include and use MinGW's
//#include "extra/mybcrypt.h"
#include <bcrypt.h>
// [ ... About 110 lines below ... ]
//if(!BCrypt_init()) return -1;  // The comment on this line in-file suggests only needed for Win98

Final Words
Could you put the batch script in an easy-to-find location... Like the page for quickbms on your site.

Author:  aluigi [ Wed Jun 05, 2019 11:44 pm ]
Post subject:  Re: About Compiling QuickBMS In Windows

Yeah compiling quickbms on Windows is complicated and it's a sort of "card castle" where everything is balanced and works on the building environment that I use for it.

Compiling on Windows is ever painful with all programs (except helloworld.c) so this is not really a surprise, various versions of VS, mingw/gcc/cygwin and so on... really painful.

1) I guess that #include is mandatory anyway. Did you cross-compile or compiled natively?

2) weird and interesting, I will review that

3) yes definitely, DO NOT TRY x86_64

4) this is interesting and also partially easy to bypass since a portion of the code from xcompress is available with source code (used on Linux)

5) this is really weird but the mybcrypt.h wrapper works well, do you get any specific error?
Here my BCrypt_Init is linked and perfectly called at runtime (just checked) and all the API referenced.
I don't have a sample ready to test and share unfortunately.

Ah, in case someone is curious... yes there are some users of quickbms who run it on Win98 :D

Author:  Brolijah [ Thu Jun 06, 2019 12:58 am ]
Post subject:  Re: About Compiling QuickBMS In Windows

(The list was just to answer any potential questions for people who come across these things as I did. Didn't mean them to sound like I was asking those questions here.)
Note 1 is just... weird. You'd think headers and libraries installed with MinGW would all be in GCC's default search path. Yet, /opt/include and /opt/lib never was for any of my installations. MinGW puts openssl in there and not the main include path. I compiled natively using MinGW-w64 GCC toolchains.
Note 4... I couldn't find much on xcompress. Didn't realize some of it was open-source.
Note 5. GCC complains about two things: (1) warning implicit declaration of BCrypt_init in myenc.c and (2) undefined reference to BCrypt_init at link time. I did think it very strange that GCC thought the call to BCrypt_init was an implicit declaration although it had already been defined with a body in the header... If you feel willing to look at the disassembly, I can attach the object file for quickbms.o. The actual function for BCrypt_init is just gone. (The easy check should've been find where "bcrypt.dll" is referenced before LoadLibrary, but the string is literally nowhere.)

Author:  Brolijah [ Thu Jun 06, 2019 1:45 am ]
Post subject:  Re: About Compiling QuickBMS In Windows

Attached below is the object file I compiled for quickbms.c using the normal code in myenc.c.
BCrypt_init should have been compiled into this object by inclusion. "quickbms.c" includes "myenc.c", and "myenc.c" includes "mybcrypt.h". Except by my search, it's nowhere. In fact, the function call is undefined according to IDA.

quickbms_temp.zip [1.8 MiB]
Downloaded 20 times

Author:  aluigi [ Thu Jun 06, 2019 7:11 pm ]
Post subject:  Re: About Compiling QuickBMS In Windows

Maybe your gcc already has __BCRYPT_H__ defined since it has been already loaded and mybcrypt.h is ignored.

Anyway with a vanilla Mingw using gcc 4 x86 on Windows (native) there are no issues and it compiles on the fly without spending hours with errors.
I remember the old t-dragon toolchain (but probably older than 2015) on which it worked perfectly too, ever on the fly using compa.bat.

Author:  Brolijah [ Mon Jun 10, 2019 9:47 pm ]
Post subject:  Re: About Compiling QuickBMS In Windows

aluigi wrote:
Maybe your gcc already has __BCRYPT_H__ defined since it has been already loaded and mybcrypt.h is ignored.

Oh god I can't believe I didn't try to test that scenario. I tried a stupid little test and I see now bcrypt is already included by another header.
In file included from C:/mingw-w64/i686-6.4.0-win32-dwarf-rt_v5-rev0/mingw32/i686-w64-mingw32/include/wincrypt.h:764:0,
                 from C:/mingw-w64/i686-6.4.0-win32-dwarf-rt_v5-rev0/mingw32/i686-w64-mingw32/include/windows.h:95,
                 from C:/mingw-w64/i686-6.4.0-win32-dwarf-rt_v5-rev0/mingw32/i686-w64-mingw32/include/winsock.h:10,
                 from C:\mingw-w64\i686-6.4.0-win32-dwarf-rt_v5-rev0\mingw32\opt\include/openssl/dtls1.h:71,
                 from C:\mingw-w64\i686-6.4.0-win32-dwarf-rt_v5-rev0\mingw32\opt\include/openssl/ssl.h:1699,
                 from src\quickbms.c:173:

That answers that, I guess. I'll update the OP to reflect this. I believe someone, someday, will stumble across this thread, and I hope it'll be of some use to them.
As a side memo, the whole reason why I set out to compile quickbms myself was because I and a couple others on Discord wanted the fixed ALLZ code built in with quickbms. Thank you again for your help and this multi-useful tool.

Author:  aluigi [ Tue Jun 11, 2019 8:51 am ]
Post subject:  Re: About Compiling QuickBMS In Windows

Wow, compiling the whole quickbms just for ALLZ?
Wasn't enough your stand-alone tool for doing that job?

Author:  Brolijah [ Tue Jun 11, 2019 7:09 pm ]
Post subject:  Re: About Compiling QuickBMS In Windows

Two reasons: It's just more convenient. Do you remember the Jump Stars ALAR archive type? That's also one of those Aqualead formats. The guys who were researching that game got stumped when they got to ALLZ from what I read on some 15 year old forum posts. Since games using ALAR are also likely using ALLZ or vice-versa, we could just use one tool for both.
The second reason is the Zanki Zero people's files are not packed in ALAR files... rather they're just ALLZ files stacked back to back. No header to them or anything, which is weird. So they need to split up those files then decompress. A time consuming process that could probably be scripted with just one tool.

Author:  aluigi [ Wed Jun 12, 2019 8:42 am ]
Post subject:  Re: About Compiling QuickBMS In Windows

Very interesting.

I hope to make a simple step-by-step for an error-free compiling once I have some free time.

After all one day I had to open this Pandora's box :D
Being the only author and maintainer of QuickBMS everything will stop once I will stop working on it, so it's necessary to have a plan B ready in place.

My only request is to avoid forks, the situation is already a mess with the older versions of the tool, therefore forks are definitely my nightmare.
I'm ready and available to implement ANY feature and crazy idea that has a sense and works (and doesn't take time to be implemented), I'm very open on that side.

Author:  aluigi [ Thu Jun 13, 2019 12:20 am ]
Post subject:  Re: About Compiling QuickBMS In Windows

As promised, this is the simple step-by-step for compiling quickbms on Windows.
There are NO requirements and it's meant for everybody.

Download gcc 4.9.4
https://sourceforge.net/projects/mingw- ... z/download

unzip gcc on your disk, maybe in c: (it will be c:\mingw32)

Download the current quickbms dependencies

unzip the include folder in c:\mingw32\i686-w64-mingw32

unzip the lib folder in c:\mingw32

download the quickbms source code and extract the src folder in c:\quickbms (it will become c:\quickbms\src)

download compa.bat and put it in c:\quickbms

open compa.bat with a text editor and:
- remove line 16: set TEMP=z:\temp
- remove line 17: set TMP=z:\temp
- replace all the occurrencies of c:\mingw with c:\mingw32

run the console (cmd.exe)

go in the quickbms directory and set the path of the Mingw compiler:
set PATH=c:\mingw32\bin


for compiling quickbms_4gb_files.exe
compa.bat -DQUICKBMS64


* quickbms 0.10.0 has a compiling bug in src\utils.c at lines 440 and 442, you can just delete them because they are unused.

* there is also the BCrypt problem reported by Brolijah, it's enough to remove line 1431 in src\myenc.c and add -lbcrypt after compa.bat:
compa.bat -DQUICKBMS64 -lbcrypt

* if you need to compile it fast for your tests remove "-s -O2" from compa.bat

* gcc 8.1.0 works perfectly too:
https://sourceforge.net/projects/mingw- ... z/download

* this step-by-step will be fixed after I will release quickbms 0.10.1

Author:  aluigi [ Thu Jun 13, 2019 1:19 am ]
Post subject:  Re: About Compiling QuickBMS In Windows

I have manually patched xcompress.lib for setting the line number field of lci.obj and ldi.obj to zero :)
The patched version is already included in quickbms_dependencies.zip
For the tech guys:
           original_xcompress.lib (MD5 8f4165d4d9a173c06bc00dc5a3ab3aab)
  Offset   |        patched_xcompress.lib
000A40D0   02       00
0010B4AC   02       00

Page 1 of 1 All times are UTC
Powered by phpBB® Forum Software © phpBB Limited