ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Sat Oct 31, 2020 7:44 pm

All times are UTC




Post new topic  Reply to topic  [ 9 posts ] 
Author Message
PostPosted: Wed Mar 18, 2020 7:23 am 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 19
Couldn't find any way to modify SQEXSEAD audio files, so I went ahead and made my own replacer tool for them :)
This was mostly used for the Dragon Quest XI Orchestral ╚░verhaul Mod, so it's only really been tested with the Dragon Quest series of games, but hopefully it should be able to support any game that uses SQEXSEAD.

SQEXSEAD is usually identified by a .sab file extension, or a "sabf" header magic - it might also be stored inside UE4 assets too (a .uasset & .uexp pair, with the sabf header inside the uexp file) - DQXI used the UE4 way (SABReplace can update the uexp & fix the uasset) but this should be able to work with non-UE4 SABs too.

IIRC you'll have best results if you encode your audio to HCA format first (or preferably HCA-MX, which requires less CPU to decode, you might need the Criware criatomencd.exe tool to encode that though)
HCA usually seems to be the main audio codec used by these, but there's some games which might not support it (and might for example require MSF audio instead - I think that's just a variant of MP3 though, so shouldn't have any trouble with that)

The SQEXSEAD format also supports OGG/WAV/ATRAC9 audio too, but sadly I never figured out how OGG was stored in the SQEXSEAD container... seemed to have something to do with each OGG chunk being referenced in the SQEXSEAD header, but I didn't manage to get very far with it so there's no support for replacing with OGGs atm.
Really it depends on the game which audio codecs will work or not - even though the SQEXSEAD format can contain all those different codecs, normally games will only include support for the codec that their audio files use, so it's usually a good idea to stick with the same one.

For more info about SQEXSEAD files you can check out the vgmstream decoder source code, was immensely helpful while creating this tool: https://github.com/losnoco/vgmstream/bl ... qex_sead.c

---

Anyway, to use SABReplace just double-click the EXE to show a small usage guide, the tool is command-line though so you'll need to run it with parameters through command-prompt.

I'll probably post the code for this up somewhere soon, if someone really wants to look at it just send me a PM.
Enjoy, if you have any questions/problems please let me know!

E: was just reminded of one small feature I forgot to post about - SABReplace will also inject a HCA loop header for you if it doesn't already exist: unfortunately most free HCA encoders don't have the ability to create this with the proper loop start/loop end values, which some games require to loop properly.
This shouldn't be a problem with the official Criware tools though, this is just for people who don't want to use those (which is why it won't touch the loop header if it exists already, the header Criware generates is probably a lot more accurate than what SABReplace can create, well the values that generates seem to be different anyway...)
You can disable this by using 0 for both loop start & loop end, if the track you're editing doesn't loop or something.


Attachments:
File comment: Added support for multiple MTRL streams, .NET Framework 4.5 required!
SABReplace-0.7a.zip [17.83 KiB]
Downloaded 80 times
File comment: Original release
SABReplace-0.6a.zip [48.24 KiB]
Downloaded 133 times


Last edited by infogram on Sun Jun 14, 2020 11:48 pm, edited 10 times in total.
Top
   
PostPosted: Sun Mar 22, 2020 10:28 pm 

Joined: Tue Mar 03, 2020 2:48 pm
Posts: 3
Hi infogram,

this is great. I'm happy you release this great piece of work of yours. I managed to get perfectly looped music in DQXI with it. As you told me before, I used Audacity to get the sample loop points your tool requires to set the header. Interestingly, I had good results using deretore's hca encoder, so no need to grab CriWare software, honestly.

All in all, this is great for me, because I can get my music into the game, eventually. =) However, it's not very end-user friendly. If we just could find a way to make those PAKs ....

I have a plan B for the case we will never figure it out, but that involves some work that would be unnecessary if we can make PAKs, plus, my user-friendly solution will limit the new music to the same size of the original pieces, because it involves re-injecting the files using quickbms.


Anyways, you're doing some great work here, infogram. Thanks!


Top
   
PostPosted: Wed May 13, 2020 12:20 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 3
First off, I'd just like to say: thanks so much for this incredible tool, Infogram! Not to mention your documentation ( https://zenhax.com/viewtopic.php?f=9&t=1005&start=540#p38382 )

Your contributions have proved invaluable to the DQXI modding community.

As it is, I've been trying to assist someone who's interested in replacing the English dialogue in the PC version with the Japanese from the Switch version. Aside from a handful of (minor) filename differences, we've encountered a warning when trying to re-inject the voice files using SABReplace:

Image

"SAB MTRL section has more than 1 entry (2)! SABReplace isn't designed to work with this!"

I was curious if there's any light you could shed on these MTRL entries (we tried checking the VGMStream source to no avail). No doubt there are still other outstanding issues when it comes to the process of replacing the dialogue. However, you're by far the best resource we would have.

I know they reached out to your collaborator (Canzah, I think?), and I believe they have been trying to reach you as well. (Might've already gotten ahold of you for all I know :lol: )

Any insight you could provide to the team would make this mod far more viable. We are all (fairly) active on Komodo's Modding Discord: https://discordapp.com/invite/hWcdCpn

Thanks :)


Top
   
PostPosted: Thu Jun 11, 2020 4:11 am 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 19
mike9k1 wrote:
[..]

Sorry for the late reply, been away from home lately, and I need to check here more often it seems..

Thanks for the kind words too :) I just saw your retro music mod as well, nice work with that!
(btw if you want to get rid of the dqxihook dependency, maybe look into https://github.com/panzi/u4pak - apparently this can create UE4 pak files for you, but I haven't tried it myself though)
(edit: oh wait, I forgot DQXI does some custom crypto stuff to the pak files, guess that might have to be modded into u4pak somehow: viewtopic.php?p=38235#p38235)

Anyway, after you get that MTRL error, does the modded file work in-game? Or does it just make it silent or something?

I got an idea why it might break it, but I don't have any .uexp files that actually have that MTRL issue to check with... if I can get hold of a uexp that gives that MTRL error maybe I can try looking into it.


Top
   
PostPosted: Fri Jun 12, 2020 5:04 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 3
infogram wrote:
(edit: oh wait, I forgot DQXI does some custom crypto stuff to the pak files, guess that might have to be modded into u4pak somehow: viewtopic.php?p=38235#p38235)

Anyway, after you get that MTRL error, does the modded file work in-game? Or does it just make it silent or something?

I got an idea why it might break it, but I don't have any .uexp files that actually have that MTRL issue to check with... if I can get hold of a uexp that gives that MTRL error maybe I can try looking into it.


With RMM, at one point I believe I tried using the "-r" (reimport) option with the quickbms script (e.g. "quickbms_4gb_files.exe -w -r dragonquest_xi_v3.bms Data_P_Retro_Music_Mod.pak JackGame\"), but had no such luck at the time. I'll have a look at the u4pak documentation, in any event. :D

I've attached a couple sample files from the Party Chat voiceovers (JackGame\Content\Sounds\Voice\PartyChat\VO_Pty_UN10801\SWAV_pty_UN10801_paha.uexp and SWAV_pty_UN10801_paha.uasset). I know these two reproduce the error.
Image

IIRC, when these are added back into the game, it crashes once called upon (the game runs fine until you try to select "Party Chat" in the menu, in this case)

In the meantime, I will try to get back in touch with D'arby (/ Raisins) on Discord to see if they've made any progress on their language mod (despite the MTRL warning). They will presumably be delighted to see that it's garnered your interest :) .


Attachments:
SWAV_pty_UN10801_paha.zip [16.34 KiB]
Downloaded 48 times
Top
   
PostPosted: Sun Jun 14, 2020 8:01 pm 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 19
mike9k1 wrote:
I've attached a couple sample files from the Party Chat voiceovers (JackGame\Content\Sounds\Voice\PartyChat\VO_Pty_UN10801\SWAV_pty_UN10801_paha.uexp and SWAV_pty_UN10801_paha.uasset). I know these two reproduce the error.

Thanks, just had a look at it and seems like I handled MTRL stuff completely wrong, originally I thought MTRL headers all came before the MTRL data itself, but it seems headers + data are actually intertwined (eg. instead of header 0, header 1, data 0, data 1, it's actually header 0, data 0, header 1, data 1...)

So when trying to update MTRL crap it'd just end up overwriting all the MTRL's data (which is the actual audio data, HCA etc), not good :)

I've added some new code which should be able to handle reading/recreating this stuff fine, in my own quick tests it seems like the data is being updated properly without overwriting now, haven't actually tried testing anything in game yet though.

Also added a new param to specify which MTRL slot to replace, got no idea what the multiple MTRLs are even there for though, you might need a good extractor to extract all of them and see which you want to replace.
(Seeing as there's two MTRLs in the voiceovers, and there's two voiceover languages IIRC, maybe they use MTRLs to handle that?)

So the new params are:
SABReplace.exe <UEXP/SAB path> <replacement file path> [loop start point] [loop end point] [MTRL slot index]
By default MTRL index will be -1, which means the last slot in the SAB, 0 would be the first slot, etc.
If you want to set the MTRL index without setting loop points just use 0 for the loop parameters.

Updated main post with 0.7 :)


Top
   
PostPosted: Mon Jun 15, 2020 1:12 pm 

Joined: Sat Sep 28, 2019 7:00 pm
Posts: 217
Just a few notes - reimport function doesn't work with scripts, where "memory_file" and "append" are used at the same time (and in most cases it doesn't work with memory_file in general, even in reimport2 / reimport3 modes). u4pak or unrealpak are producing default pak format, incompatible with the game. Creators of Orchestral Mod are using their own custom solution (like modified u4pak or something) for repacking stuff into compatible paks with all those xor, not, etc. Not sure if those tools are available for public, but I didn't search much.

_________________
You can request AES keys on rin forums (the list with keys is also there)
AES keys finder and latest UE4 bms scripts: in this post


Top
   
PostPosted: Thu Jul 09, 2020 8:31 pm 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 19
So thanks to the FFXV debug build we now know pretty much 100% of the SAB structure (well up to the version used in FFXV at least, there's been some changes seen in other games since then, but AFAIK those are pretty minor)

I've been working on a new SAB tool that can convert those structures into XML, and vice-versa, letting you tweak a ton of internal SAB parameters.
Here's an example converted SAB from FFXV (bgm_car_ff13or_01.win.sab): https://pastebin.com/dA6aV0FY

Got the conversion stuff mostly working, still need to work on some DataFileName-related stuff though (automatically detecting type of data: HCA/PCM/Vorbis/etc, updating with sample rate/channel count/loop data from that file, etc), hopefully I'll have something to release soon :)

E: oh damn, only just found out that .MAB files are also used for storing music data like SABs do.. I saw a few things about MABs before but assumed it was just a MIDI-kinda thing (which it does seem to be), but in FFXV a lot of BGM tracks get stored using it too...
Not really sure why they didn't just use SAB since they seem to be skipping all the useful things MAB files are used for, now I guess this means I'll have to look into adding MAB stuff too, seems pretty complicated though :/

spiritovod wrote:
Creators of Orchestral Mod are using their own custom solution (like modified u4pak or something) for repacking stuff into compatible paks with all those xor, not, etc. Not sure if those tools are available for public, but I didn't search much.

That was thanks to somebody offering to create the paks for us, we never actually got given any tools for it or anything :(
IIRC the person who made it was part of some TL team, and I guess they were scared of someone beating them to translating or something.
That was years ago now though, kinda sad to see they still never released their tools...

On that note, I did try updating UnrealPak with DQXI's custom stuff during Orchestal's development, IIRC it was creating paks fine but we couldn't figure out how to get UnrealPak to use the correct folder path in the pak header, or something like that... like I said years ago, and my memory is awful :P
Don't have any of the code for that anymore, but I don't think it was too hard to add the custom DQXI stuff to it.


Top
   
PostPosted: Fri Jul 24, 2020 12:23 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 3
Hi Infogram!

Well, we never ended up making any additional progress on the voices mod, but it's kind of a moot point now, as you're probably already aware: it's been announced that Definitive Edition will be coming to PC on December 4th.

We've actually begun using DQXIHook for injecting other mods (DataTables, etc) in the base game, and making some use of the console functions as well. It's turned into quite the tool :D

But of course, now there's concern over portability to the "S" version when it comes out.

When it rolls around, do you think that it'd be worth updating DQXIHook at that point? I can certainly see how there'd be a lot less motivation now, without the need for OOM.

If not, here's hoping we can figure out the u4pak method at some point.

Cheers! ;)


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 9 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited