Skip to content

Commit f19f023

Browse files
committed
fix(dat): move dat read/decompress/decode off the render thread
QueueDecode ran the whole pipeline - dat mapping, MFT parse, custom Huffman decompression, DXT decode - inline inside the DX task, which is drained synchronously on the game's render thread once per frame. A cold first-time MFT parse (~150k-entry hash table) or just a big/ compressed texture would visibly stall the game. Only the final GPU upload actually needs the D3D device; move the rest onto the existing worker thread pool (already used elsewhere in Resources) and only hop back to the DX queue for MakeTextureFromArgb. Also reserve the fileid_to_slot map before filling it and index rather than range-for over the MFT hash table - in a debug build (checked iterators), that loop over ~150k+ entries was a real chunk of the one-time parse cost.
1 parent a39302f commit f19f023

1 file changed

Lines changed: 21 additions & 9 deletions

File tree

GWToolboxdll/Modules/GwDatModule.cpp

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -240,19 +240,26 @@ namespace {
240240
// failure (missing dat data) rather than merely not-yet-decoded.
241241
bool HasFailed(const GwImg* img) { return img->m_completed && !img->m_tex; }
242242

243-
// Reads + decodes + uploads on the DX thread (the only place we have the device). GwImg is shared-owned,
244-
// so the task keeps its image alive even if Terminate() clears the cache before it runs.
243+
// Reads + decompresses + decodes on a worker thread (mapping/MFT reads are safe from any thread once
244+
// loaded, per EnsureLoaded/ParseIndex) - this is the slow part (a cold dat-mapping/MFT parse, or just a
245+
// big/compressed file) and previously ran inline on the render thread, stalling the game for as long as
246+
// it took. Only the final GPU upload needs the D3D device, which only the render thread may touch, so
247+
// that alone hops to the DX task queue. GwImg is shared-owned, so both tasks keep it alive even if
248+
// Terminate() clears the cache before they run.
245249
template <typename Decode>
246250
void QueueDecode(std::shared_ptr<GwImg> img, Decode decode)
247251
{
248252
img->m_pending = true;
249-
Resources::Instance().EnqueueDxTask([img, decode](IDirect3DDevice9* device) {
253+
Resources::Instance().EnqueueWorkerTask([img, decode] {
250254
std::vector<uint32_t> argb;
251255
Vec2i dims;
252-
if (decode(argb, dims) && dims.x > 0 && dims.y > 0)
253-
img->m_tex = MakeTextureFromArgb(device, argb, dims);
254-
img->m_dims = dims;
255-
img->m_completed = true;
256+
const bool ok = decode(argb, dims) && dims.x > 0 && dims.y > 0;
257+
Resources::Instance().EnqueueDxTask([img, argb, dims, ok](IDirect3DDevice9* device) {
258+
if (ok)
259+
img->m_tex = MakeTextureFromArgb(device, argb, dims);
260+
img->m_dims = dims;
261+
img->m_completed = true;
262+
});
256263
});
257264
}
258265

@@ -557,8 +564,13 @@ bool GwDatModule::ParseFrom(void* mapping_v, long long file_size, std::vector<Mf
557564
expansion_count * sizeof(MftExpansion)))
558565
return false;
559566

560-
// Map each file id to its base slot; several ids may alias the same slot.
561-
for (const auto& e : mftx) {
567+
// Map each file id to its base slot; several ids may alias the same slot. Reserve up front (this
568+
// is a ~150k+ entry table) so the loop doesn't rehash repeatedly, and index rather than range-for
569+
// over `mftx` - in a debug build (checked iterators) that's a measurable chunk of the one-time
570+
// parse cost, which previously ran inline on the render thread.
571+
fileid_to_slot.reserve(static_cast<size_t>(expansion_count));
572+
for (int i = 0; i < expansion_count; ++i) {
573+
const MftExpansion& e = mftx[static_cast<size_t>(i)];
562574
if (e.file_offset >= 16 && e.file_offset < slot_count)
563575
fileid_to_slot[static_cast<uint32_t>(e.file_number)] = e.file_offset;
564576
}

0 commit comments

Comments
 (0)