ZenHAX

Free Game Research Forum | Official QuickBMS support | twitter @zenhax | SSL HTTPS://zenhax.com
It is currently Fri Oct 22, 2021 1:04 am

All times are UTC




Post new topic  Reply to topic  [ 14 posts ] 
Author Message
PostPosted: Wed Mar 18, 2020 7:23 am 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 20
Couldn't find any way to modify SQEXSEAD audio files, so I went ahead and made my own replacer tool for them :)
This was mostly used for the Dragon Quest XI Orchestral ╚░verhaul Mod, so it's only really been tested with the Dragon Quest series of games, but hopefully it should be able to support any game that uses SQEXSEAD.

SQEXSEAD is usually identified by a .sab file extension, or a "sabf" header magic - it might also be stored inside UE4 assets too (a .uasset & .uexp pair, with the sabf header inside the uexp file) - DQXI used the UE4 way (SABReplace can update the uexp & fix the uasset) but this should be able to work with non-UE4 SABs too.

IIRC you'll have best results if you encode your audio to HCA format first (or preferably HCA-MX, which requires less CPU to decode, you might need the Criware criatomencd.exe tool to encode that though)
HCA usually seems to be the main audio codec used by these, but there's some games which might not support it (and might for example require MSF audio instead - I think that's just a variant of MP3 though, so shouldn't have any trouble with that)

The SQEXSEAD format also supports OGG/WAV/ATRAC9 audio too, but sadly I never figured out how OGG was stored in the SQEXSEAD container... seemed to have something to do with each OGG chunk being referenced in the SQEXSEAD header, but I didn't manage to get very far with it so there's no support for replacing with OGGs atm.
Really it depends on the game which audio codecs will work or not - even though the SQEXSEAD format can contain all those different codecs, normally games will only include support for the codec that their audio files use, so it's usually a good idea to stick with the same one.

For more info about SQEXSEAD files you can check out the vgmstream decoder source code, was immensely helpful while creating this tool: https://github.com/losnoco/vgmstream/bl ... qex_sead.c

---

Anyway, to use SABReplace just double-click the EXE to show a small usage guide, the tool is command-line though so you'll need to run it with parameters through command-prompt.

I'll probably post the code for this up somewhere soon, if someone really wants to look at it just send me a PM.
Enjoy, if you have any questions/problems please let me know!

E: was just reminded of one small feature I forgot to post about - SABReplace will also inject a HCA loop header for you if it doesn't already exist: unfortunately most free HCA encoders don't have the ability to create this with the proper loop start/loop end values, which some games require to loop properly.
This shouldn't be a problem with the official Criware tools though, this is just for people who don't want to use those (which is why it won't touch the loop header if it exists already, the header Criware generates is probably a lot more accurate than what SABReplace can create, well the values that generates seem to be different anyway...)
You can disable this by using 0 for both loop start & loop end, if the track you're editing doesn't loop or something.


Attachments:
File comment: Added support for multiple MTRL streams, .NET Framework 4.5 required!
SABReplace-0.7a.zip [17.83 KiB]
Downloaded 519 times
File comment: Original release
SABReplace-0.6a.zip [48.24 KiB]
Downloaded 305 times


Last edited by infogram on Sun Jun 14, 2020 11:48 pm, edited 10 times in total.
Top
   
PostPosted: Sun Mar 22, 2020 10:28 pm 

Joined: Tue Mar 03, 2020 2:48 pm
Posts: 3
Hi infogram,

this is great. I'm happy you release this great piece of work of yours. I managed to get perfectly looped music in DQXI with it. As you told me before, I used Audacity to get the sample loop points your tool requires to set the header. Interestingly, I had good results using deretore's hca encoder, so no need to grab CriWare software, honestly.

All in all, this is great for me, because I can get my music into the game, eventually. =) However, it's not very end-user friendly. If we just could find a way to make those PAKs ....

I have a plan B for the case we will never figure it out, but that involves some work that would be unnecessary if we can make PAKs, plus, my user-friendly solution will limit the new music to the same size of the original pieces, because it involves re-injecting the files using quickbms.


Anyways, you're doing some great work here, infogram. Thanks!


Top
   
PostPosted: Wed May 13, 2020 12:20 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 9
First off, I'd just like to say: thanks so much for this incredible tool, Infogram! Not to mention your documentation ( https://zenhax.com/viewtopic.php?f=9&t=1005&start=540#p38382 )

Your contributions have proved invaluable to the DQXI modding community.

As it is, I've been trying to assist someone who's interested in replacing the English dialogue in the PC version with the Japanese from the Switch version. Aside from a handful of (minor) filename differences, we've encountered a warning when trying to re-inject the voice files using SABReplace:

Image

"SAB MTRL section has more than 1 entry (2)! SABReplace isn't designed to work with this!"

I was curious if there's any light you could shed on these MTRL entries (we tried checking the VGMStream source to no avail). No doubt there are still other outstanding issues when it comes to the process of replacing the dialogue. However, you're by far the best resource we would have.

I know they reached out to your collaborator (Canzah, I think?), and I believe they have been trying to reach you as well. (Might've already gotten ahold of you for all I know :lol: )

Any insight you could provide to the team would make this mod far more viable. We are all (fairly) active on Komodo's Modding Discord: https://discordapp.com/invite/hWcdCpn

Thanks :)


Top
   
PostPosted: Thu Jun 11, 2020 4:11 am 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 20
mike9k1 wrote:
[..]

Sorry for the late reply, been away from home lately, and I need to check here more often it seems..

Thanks for the kind words too :) I just saw your retro music mod as well, nice work with that!
(btw if you want to get rid of the dqxihook dependency, maybe look into https://github.com/panzi/u4pak - apparently this can create UE4 pak files for you, but I haven't tried it myself though)
(edit: oh wait, I forgot DQXI does some custom crypto stuff to the pak files, guess that might have to be modded into u4pak somehow: viewtopic.php?p=38235#p38235)

Anyway, after you get that MTRL error, does the modded file work in-game? Or does it just make it silent or something?

I got an idea why it might break it, but I don't have any .uexp files that actually have that MTRL issue to check with... if I can get hold of a uexp that gives that MTRL error maybe I can try looking into it.


Top
   
PostPosted: Fri Jun 12, 2020 5:04 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 9
infogram wrote:
(edit: oh wait, I forgot DQXI does some custom crypto stuff to the pak files, guess that might have to be modded into u4pak somehow: viewtopic.php?p=38235#p38235)

Anyway, after you get that MTRL error, does the modded file work in-game? Or does it just make it silent or something?

I got an idea why it might break it, but I don't have any .uexp files that actually have that MTRL issue to check with... if I can get hold of a uexp that gives that MTRL error maybe I can try looking into it.


With RMM, at one point I believe I tried using the "-r" (reimport) option with the quickbms script (e.g. "quickbms_4gb_files.exe -w -r dragonquest_xi_v3.bms Data_P_Retro_Music_Mod.pak JackGame\"), but had no such luck at the time. I'll have a look at the u4pak documentation, in any event. :D

I've attached a couple sample files from the Party Chat voiceovers (JackGame\Content\Sounds\Voice\PartyChat\VO_Pty_UN10801\SWAV_pty_UN10801_paha.uexp and SWAV_pty_UN10801_paha.uasset). I know these two reproduce the error.
Image

IIRC, when these are added back into the game, it crashes once called upon (the game runs fine until you try to select "Party Chat" in the menu, in this case)

In the meantime, I will try to get back in touch with D'arby (/ Raisins) on Discord to see if they've made any progress on their language mod (despite the MTRL warning). They will presumably be delighted to see that it's garnered your interest :) .


Attachments:
SWAV_pty_UN10801_paha.zip [16.34 KiB]
Downloaded 185 times
Top
   
PostPosted: Sun Jun 14, 2020 8:01 pm 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 20
mike9k1 wrote:
I've attached a couple sample files from the Party Chat voiceovers (JackGame\Content\Sounds\Voice\PartyChat\VO_Pty_UN10801\SWAV_pty_UN10801_paha.uexp and SWAV_pty_UN10801_paha.uasset). I know these two reproduce the error.

Thanks, just had a look at it and seems like I handled MTRL stuff completely wrong, originally I thought MTRL headers all came before the MTRL data itself, but it seems headers + data are actually intertwined (eg. instead of header 0, header 1, data 0, data 1, it's actually header 0, data 0, header 1, data 1...)

So when trying to update MTRL crap it'd just end up overwriting all the MTRL's data (which is the actual audio data, HCA etc), not good :)

I've added some new code which should be able to handle reading/recreating this stuff fine, in my own quick tests it seems like the data is being updated properly without overwriting now, haven't actually tried testing anything in game yet though.

Also added a new param to specify which MTRL slot to replace, got no idea what the multiple MTRLs are even there for though, you might need a good extractor to extract all of them and see which you want to replace.
(Seeing as there's two MTRLs in the voiceovers, and there's two voiceover languages IIRC, maybe they use MTRLs to handle that?)

So the new params are:
SABReplace.exe <UEXP/SAB path> <replacement file path> [loop start point] [loop end point] [MTRL slot index]
By default MTRL index will be -1, which means the last slot in the SAB, 0 would be the first slot, etc.
If you want to set the MTRL index without setting loop points just use 0 for the loop parameters.

Updated main post with 0.7 :)


Top
   
PostPosted: Mon Jun 15, 2020 1:12 pm 

Joined: Sat Sep 28, 2019 7:00 pm
Posts: 436
Just a few notes - reimport function doesn't work with scripts, where "memory_file" and "append" are used at the same time (and in most cases it doesn't work with memory_file in general, even in reimport2 / reimport3 modes). u4pak or unrealpak are producing default pak format, incompatible with the game. Creators of Orchestral Mod are using their own custom solution (like modified u4pak or something) for repacking stuff into compatible paks with all those xor, not, etc. Not sure if those tools are available for public, but I didn't search much.

_________________
You can request AES keys on rin forums (the list with keys is also there)
AES keys finder and latest UE4 bms scripts: in this post


Top
   
PostPosted: Thu Jul 09, 2020 8:31 pm 

Joined: Fri Nov 03, 2017 8:31 pm
Posts: 20
So thanks to the FFXV debug build we now know pretty much 100% of the SAB structure (well up to the version used in FFXV at least, there's been some changes seen in other games since then, but AFAIK those are pretty minor)

I've been working on a new SAB tool that can convert those structures into XML, and vice-versa, letting you tweak a ton of internal SAB parameters.
Here's an example converted SAB from FFXV (bgm_car_ff13or_01.win.sab): https://pastebin.com/dA6aV0FY

Got the conversion stuff mostly working, still need to work on some DataFileName-related stuff though (automatically detecting type of data: HCA/PCM/Vorbis/etc, updating with sample rate/channel count/loop data from that file, etc), hopefully I'll have something to release soon :)

E: oh damn, only just found out that .MAB files are also used for storing music data like SABs do.. I saw a few things about MABs before but assumed it was just a MIDI-kinda thing (which it does seem to be), but in FFXV a lot of BGM tracks get stored using it too...
Not really sure why they didn't just use SAB since they seem to be skipping all the useful things MAB files are used for, now I guess this means I'll have to look into adding MAB stuff too, seems pretty complicated though :/

spiritovod wrote:
Creators of Orchestral Mod are using their own custom solution (like modified u4pak or something) for repacking stuff into compatible paks with all those xor, not, etc. Not sure if those tools are available for public, but I didn't search much.

That was thanks to somebody offering to create the paks for us, we never actually got given any tools for it or anything :(
IIRC the person who made it was part of some TL team, and I guess they were scared of someone beating them to translating or something.
That was years ago now though, kinda sad to see they still never released their tools...

On that note, I did try updating UnrealPak with DQXI's custom stuff during Orchestal's development, IIRC it was creating paks fine but we couldn't figure out how to get UnrealPak to use the correct folder path in the pak header, or something like that... like I said years ago, and my memory is awful :P
Don't have any of the code for that anymore, but I don't think it was too hard to add the custom DQXI stuff to it.


Top
   
PostPosted: Fri Jul 24, 2020 12:23 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 9
Hi Infogram!

Well, we never ended up making any additional progress on the voices mod, but it's kind of a moot point now, as you're probably already aware: it's been announced that Definitive Edition will be coming to PC on December 4th.

We've actually begun using DQXIHook for injecting other mods (DataTables, etc) in the base game, and making some use of the console functions as well. It's turned into quite the tool :D

But of course, now there's concern over portability to the "S" version when it comes out.

When it rolls around, do you think that it'd be worth updating DQXIHook at that point? I can certainly see how there'd be a lot less motivation now, without the need for OOM.

If not, here's hoping we can figure out the u4pak method at some point.

Cheers! ;)


Top
   
PostPosted: Tue Dec 15, 2020 2:25 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 9
I just wanted to check in on this thread, since you've all been so helpful.

We managed to get pak files working, thanks to spiritovod. I did find that using SABReplace on the Definitive Edition music files resulted in a few bytes being out of place, which is easily addressed and thought I would make a note of it here

uasset: copy 3 bytes starting @ (around) 0x240 - 0x242 to 0x230 - 0x232, zero out 0x240 to 0x242 -- this actually depends on the length of the file, so I use an offset that is -92 and -76 from the end of the file respectively
uexp:
1. copy bytes starting @ 0x035 - 0x044 to 0x031 - 0x040, insert a value of "01" to 0x041 and zero out 0x042 - 0x044
2. copy 3 bytes starting @ 0x04D to 0x04F to 0x049 - 0x04B, zero out 0x04D to 0x04F

I have a couple scripts (attached) that does exactly this. I'm sure it could be a lot better, but maybe these notes could come in handy.

Thanks! :)


Attachments:
File comment: 2 scripts:
-sabr-dqxis-de-fix-uasset.ps1
-sabr-dqxis-de-fix-uexp.ps1

sabr-dqxis-de-fix.zip [1.04 KiB]
Downloaded 161 times
Top
   
PostPosted: Wed Feb 10, 2021 6:17 pm 

Joined: Fri May 20, 2016 9:13 pm
Posts: 13
Hello, sorry to resurrect a thread, but I have a problem relating to .mab files and SQEXSEAD from Balan Wonderworld. I'm trying to start modding stuff on the demo before the full game is out, and among other things I've been looking into modding the music. It's .hca wrapped in .mab which in turn is wrapped in the ue4 stuff (.uexp/uasset). I've managed to replace a short jingle using hex editing as well as a stage song. I had to edit some additional values in the files so that the stage song wouldn't cut off. But here's the problem, I have done tests with both custom looped .hca and .hca that have looping from start to end. Only the start-to-end work. Even though the game uses custom loop points in its .hca. And by work, I mean they play once, the looping never kicks in. As for the custom looped ones I made, they don't play at all. It's just silence. Even though I edited the values in the .uexp file that correspond to loop start and end to match my custom file, it just doesn't work. I used criatomencd to convert my .wavs. Neither HCA codec or HCA-MX made much difference. The custom looped hca refuse to work and I'm at my wits end here.

https://drive.google.com/file/d/1TjmOH6 ... sp=sharing
Here's some .uexp/uasset files as well as two custom .hca songs I used for testing. I've added three pairs of ue4 wrapped songs, because they all have their differences. One is for the main hub. And is a more standard .mab in the game. Grassland one is for the first chapter stage in the demo/game and starting from 0x250 till like 0xD10 has a lot of bytes which I don't exactly know what they are for. All the other stage songs mabs have such bytes. One stage uses two different songs so it has two hcas packed inside the mab. But Grassland just uses one. If needed I'll add that one too. And lastly a short jingle, it has inside a .hca that doesn't loop.
I know this is getting long, but from my additional testing: I replaced an entire mab section from grassland .uexp with the one from main hub. The game didn't complain and played the song as it should.
I would appreciate any kind of help, even if it's just a nudge closer to what I'm doing wrong here. Thanks.


Top
   
PostPosted: Thu Apr 01, 2021 8:26 pm 

Joined: Thu Sep 28, 2017 7:58 pm
Posts: 10
Sorry for the bump, could this work for uassets without an uexp pair? From Kingdom Hearts 0.2 PS4.


Attachments:
Rebellis.7z [1.08 MiB]
Downloaded 90 times
Top
   
PostPosted: Mon Aug 23, 2021 11:25 pm 

Joined: Fri Aug 06, 2021 9:07 pm
Posts: 8
Could this help with the FFVIIR Voice Over files?


Top
   
PostPosted: Wed Oct 13, 2021 5:16 am 

Joined: Sat Feb 15, 2020 1:47 am
Posts: 9
Skyryan wrote:
Could this help with the FFVIIR Voice Over files?

It is possible. It would most likely need some modifications, however.

I will tell you that the voice files for DQXI turned out to be a lot trickier than anticipated. Each .sab in the Voice\ folder actually contained several different streams with each stream being a single voice line. A couple examples:

Sylvando_Accl - might contain 8 different voice lines for the same in-game action (most likely chosen at random)
B_110_CS_230 - might contain 50 different voice lines (every single voice line for every different character in the same cutscene)

My guess is that the .mab files that FF7R uses might be a similar beast, with multiple streams for different voice lines. I vaguely recall seeing some modified SABReplace for FF7R so you might look around? I would be rather surprised if no tools for audio replacement existed by this point. It is a rather popular title.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 14 posts ] 

All times are UTC


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Limited