I have a suggestion.
Can we keep TOC and all asset chunks SEPARATED in real device
(when the game engine accepts uncompressed/unencrypted asset chunks),
and make a virtual file which (virtually) merges them together?
(that means, for example, copy /b TOC.0+chunk.1+chunk.2+...+chunk.999 arc.bin on REAL device, is same as that VIRTUALIZED arc.bin)
The reason is,
most game archives nowdays are made to stuff bluray disc, they're very unfriendly to PC storage.
If we keep asset chunks separated, we can hardlink all the duplicates and apply OS native compression on specific type of asset.
In Yakuza0, all archives (*.par) sums up 13.9GB,
after unpack, hardlink the duplicates, and apply "compact /C /I /S /EXE:LZX", it results 6.7GB.
Also many Unity indies have a lot duplicated large textures in their game archive.
I know this can be done with a kernel32 proxy dll, with CreateFile, ReadFile, SetFilePointer etc. hooked,
But I am looking for a more generalized solution like Dokan/Dokany (github.com/dokan-dev/dokany) , pfmap (pismotec.com/download) or CBFS (http://www.callbacktechnologies.com/cbfsconnect
I actually made a QuickBMS-fs with Dokan before, which similar to: github.com/dorvan/chunking-filesystem,
for random accessing without extracting the whole archive (lost those codes as my old laptop hard drive broke).
Now I hope this can be supported natively. (in reverse way)
I think keep separated/split chunks on real hard disk, and feed the game executable with a virtually merged archive,
can fulfill these two purposes:
1. Reduce disk space waste on PC due to the console/BD disc-affiliated archiving.
(duplicated asset across archives, both compressible and incompressible type of resource in one archive ,etc)
2. Avoid repeatedly packing/unpacking when moding, which can actually harm your hard drive.