A frame that allocates nothing

A frame has a budget. At 240 Hz it is 4.1 ms, and the part that hurts is not the average cost of a heap allocation — it is the tail. Usually new is a few hundred nanoseconds. Occasionally it takes a lock another thread already holds, or grows the heap, or faults in a page, and the frame that happens to be holding the bag drops.

This engine makes that worse than usual, because it is genuinely concurrent: four AI worker threads and the XAudio2 mixer thread all allocate against the same CRT heap as the game thread. Contention is a real mechanism here, not a textbook one. The same heap lock turns up as the reason a force-terminated worker can deadlock the whole process — see Monster AI on its own threads.

So the rule, written into docs/ARCHITECTURE.md:

Steady-state frames perform no heap allocation.

Five patterns, not one

The interesting half of that rule is that it is not implemented by one technique applied everywhere. Five subsystems allocate in five different shapes, and the shape decides the pattern.

The arena. gfx::UploadAllocator is a bump pointer over a persistently mapped upload-heap buffer. Allocate is a few instructions; Reset reclaims everything at once. Every per-draw constant buffer, skinning palette, and UI vertex comes from it. There is one allocator per frame in flight — with kFrameCount = 3, the GPU can still be reading frame N's data while the CPU writes frame N+1's. Three consumers hold their own: the renderer and the sprite batch take 4 MB each, particles 1 MB, so 27 MB of upload memory is reserved up front and never grows.

The price is a rule you cannot enforce in the type system: an allocation is valid for exactly one frame. Cache a returned pointer and it will be overwritten by next frame's traffic.

The pool. Audio playback used to heap-allocate a voice object, create an OS source voice, and copy the entire sample buffer — on every single Play. Now source voices are pooled and reused by sample format, capped at 32, and playback references the caller's PCM memory directly. Once the pool is warm, Play allocates nothing and copies nothing.

Flat grids instead of node containers. The AI's per-frame snapshot is pooled, but pooling a container is worthless if the container frees its storage on clear(). This one is covered in detail in the async AI entry; the short version is that unordered_set::clear() frees its nodes, so every re-publish re-allocated them — roughly 15–20 allocations per frame from a buffer that was supposed to be recycled.

The free list. AllocateSrv was a monotonic bump allocator over the 1024-slot shader-visible CBV/SRV heap, and slots were never returned. Every texture-churn path leaked: font-atlas rebakes on a window resize, level-transition palette reloads, quality hot-swaps, turbidity rebuilds. A long session would eventually trip DN_ASSERT("SRV heap exhausted") — which aborts in release too. Now gfx::Texture returns its slot on destruction and AllocateSrv reuses freed slots LIFO before bumping the high-water mark, so the heap bounds the live texture count rather than the total ever created.

Retained capacity. The light list, the sprite batch's vertices, the animator's pose and palette buffers are long-lived members that get cleared and refilled, never destroyed — reserved up front (the sprite batch takes 8,192 vertices) so they are de-facto pools of their own elements. And HUD label strings are reformatted only when the underlying value changes, never once per frame just because the frame came round again.

What it cost

The optimisation created a use-after-free. Zero-copy audio playback means a submitted buffer points into the caller's sample memory. AudioEngine is constructed before Game in Main, so destruction runs the other way: ~Game freed the sound-bank buffers before ~AudioEngine destroyed the voices. A sound still playing at exit left the XAudio2 mixer thread reading freed memory. The fix is StopAll(), called from ~Game, which destroys every voice — DestroyVoice blocks until the mixer thread lets go — so no reference into Game-owned memory survives that memory.

That is the honest shape of this work: removing the copy moved a lifetime problem from the allocator into the shutdown order, and construction order became load-bearing in a way it had not been before.

Descriptor recycling created a drain requirement. A recycled slot's old descriptor can still be referenced by up to two in-flight frames, so whoever overwrites one must drain the GPU first. Texture::Upload already blocked via ExecuteImmediate; Texture::RenderTarget had to gain a WaitIdle.

Writing the rule down did not enforce it. Three weeks after the rule landed in ARCHITECTURE.md, a deliberate audit found three paths violating it every frame: a formation pass building three fresh vectors, the AI snapshot's node containers, and an animated icon bake rebuilding its four-light rig each time. The second of those is the one worth dwelling on — the pool's own comment claimed clear() retained capacity, and for those container types it was simply false. The code was documented as doing something it did not do, which is worse than being undocumented.

Load-time deliberately opts out. Mesh and image vectors, D3D resource creation, the one-shot upload path — all plain ownership, because they run once at startup where clarity beats allocator ceremony. C-API boundaries (cgltf, FILE*, shell COM) ride RAII wrappers so an exception mid-parse cannot leak, but nothing there is pooled. A rule applied everywhere would have made the loaders worse for no measurable gain.

The first measurement I published was wrong. When the load-time table finally existed I reported it with the note that the times were debug-inflated but the allocation counts were not. They are. The same level load counts 129,000 allocations in Debug against 41,000 in Release, and the mechanism is worth knowing: MSVC's iterator debugging gives std::vector a move constructor that allocates an iterator-debug proxy, which makes it not noexcept, which makes move_if_noexcept copy on every push_back re-allocation — and each copied animation channel re-allocates both of its buffers. A number you cannot trust is worse than no number, and this one was mine.

What it bought

• Frame time stopped depending on the allocator, and stopped competing for the heap lock with four AI workers and the mixer thread.
• Play allocates and copies nothing once the pool is warm — previously it did both, per sound, per trigger.
• The SRV heap stopped leaking. With alloc/free logging in place: boot reaches the menu on 38 slots and a loaded level on 239; two window resizes freed 14 font-atlas slots and recycled 13; a Medium→High quality swap freed 30 and recycled all 30 with zero fresh allocations, leaving the high-water mark pinned at 239 where each swap previously leaked about thirty.
• Shutdown got safer, which was not the goal but was the most valuable thing the audit produced.
• Loading got cheaper by two lines. Chasing that debug-vs-release gap led back to the glTF loader, where the clip and channel vectors grew without being reserved — so every re-allocation copied what it held. Two reserve calls took a rigged monster from 47,725 allocations to 24,332, and a whole level load from 223,000 to 129,000. The remaining Debug overhead is the per-move proxy itself, which no amount of care in our own code removes.

How it's proved

Better than it was, and the weak parts are still worth naming.

There is now an automated guard. Core/AllocTrack replaces the global operator new family and counts into a per-thread, constant-initialised slot — no lock, no allocation, nothing to re-enter. A frame guard arms on frames the rule actually covers — playing, no load, no console, no overlay, and settled that way for 120 frames — and a frame that allocates gets its call stacks symbolised into the log, each unique site once per session.

It found three per-frame allocations the moment it ran. A const std::string& bound to a ternary whose other arm was "": the common type of std::string and const char* is std::string, so the reference bound to a copy of the selected item — once per dropdown, per frame, in a draw path. A fresh byte buffer for every GPU-counter sample. And an animation clip name returned by value, so every monster state change allocated. After those, 21,338 armed frames with the party standing still allocate nothing, and the four AI workers total 8–50 allocations for a whole session.

tools/AllocTest.ps1 drives the whole run — launch, load a level, measure, exit non-zero on failure — and allocpoke plus a -SelfTest switch invert the expected verdict, so the harness has to catch a deliberate violation before it is allowed to pass. A regression test that cannot fail proves nothing.

What it still does not do is run without me, and it watches the main thread's frame rather than each worker's tick. tools/ThreadStress still carries the AI pools under load.

Still open

All three items that used to sit here have been dealt with — the counter built, the ceiling instrumented rather than removed, the load-time cost measured. What follows is what that work left behind, which is a shorter and more specific list than the one it replaced.

• Nothing runs the test but me. tools/AllocTest.ps1 launches the game, starts a level, measures a window of genuinely steady frames and exits non-zero on failure — a real regression run, with allocpoke and a -SelfTest switch that inverts the verdict so the harness has to catch a deliberate violation to pass. But it drives a live window, so it is a command someone remembers rather than a gate that blocks a merge. The rule is enforced by habit again; just a very much cheaper habit.
• A worker's tick is not guarded. The four AI threads are counted, and their whole-session totals come in at 8–50 allocations each, which is the snapshot and plan pools doing exactly what they were built for. But that is a total, not a per-tick assertion: a regression inside a worker would show up as a slowly rising number nobody is watching, where the same regression on the main thread now names its own call stack.
• 1024 descriptor slots is still a ceiling — now a visible one. Live and peak occupancy ride the dev console's perf panel, 75% and 90% log a warning on the crossing, and the exhaustion assert quotes the peak so it reads as something is leaking rather than the limit is 1024. It is still an abort. Growing the heap needs index-only descriptor handles first, because the absolute CPU/GPU pointers handed out today would dangle across a reallocation — and the measurement says that refactor has not earned itself: the showcase level sits at 275 of 1024, and two full quality swaps leave live and peak unmoved at 275. Deferred with a number behind it rather than a shrug.
• An event frame still allocates. Walk into a wall and the world raises a localised message; loc::Tr and the message log each build a string. That is allocation proportional to events, not to frames, so the guard reports it and does not assert on it. It is also not wrapped in an exemption scope, because an exemption there would equally hide the bug where something starts logging every frame. That is a judgement call, and it is the one I would expect to be argued with.
• Load-time has a floor I have not broken. The measurement immediately named its own answer: 80% of a level load's allocations are four rigged skeletons, each 40 animation clips over 33 joints, and every one of those 3,960 channels needs a times buffer and a values buffer. 7,920 allocations per model is the floor for that layout. Flattening a clip's channels into a single arena and teaching the sampler to index it would break the floor, and I do not think it is worth it for a path that runs once per level — but that is the shape of the remaining cost, not a mystery any more.