Monster AI on its own threads

Pathfinding is the most expensive thing a dungeon crawler does per monster, and it is spiky. A monster standing in a corridor costs nothing; the same monster, the moment the party rounds a corner, pays a breadth-first search that in the worst case visits every walkable cell on the map. With a dozen monsters awake, those searches land in the same frame and the frame is gone.

The reflex is to optimise the search. The better question is why the search is on the main thread at all — and once asked, a second follows immediately: what is actually unsafe about moving it off?

The decision

Split THINK from ACT. Move THINK to worker threads, keep ACT on the main thread, and make thinking pure so the split is safe by construction.

• THINK decides a monster's standing orders — an Intent (idle, engage, kite, flee) plus a full chase path. It reads an immutable snapshot of the world through a small read-only interface and writes only to its outputs. It touches no live game state.
• ACT pops path cells, re-validates each against live occupancy, commits the step, resolves the attack. It runs every frame, on the main thread, serially.

The main thread publishes one snapshot per frame — cheap — and consumes whatever plans the workers have finished. All world mutation stays in one place. The workers never touch it.

The load-bearing constraint is purity, not locking. Brain::Think and Brain::FindPath are const and read only their arguments. That is what licenses the threading; the mutexes in the handoff are incidental, and only ever held for the length of a shared_ptr swap.

The bucketing, which was a design idea first

Monsters carry an iq stat. It was in the catalog before any of this existed, as flavour: a skeleton is dim, a lich is not. Bucketing thinking by IQ turns that flavour into the scheduling strategy —

— and the mechanism disappears into the fiction. A stupid monster re-plans twice a second and a clever one four times a second, which is exactly what "stupid" ought to mean.

Acting is not bucketed. Move and attack speed stay governed by each monster's own cooldowns, so a dim monster still walks and swings at full speed. Only its change of mind lags. Between plan updates it keeps executing its cached path, which is precisely what decouples reaction time from movement speed — and, incidentally, what makes a slow bucket cheap rather than broken.

Cicada cadences

The first cut used powers of two: 0.25 / 0.5 / 1 / 2 seconds. Harmonic — their lowest common multiple is two seconds, so every two seconds all four buckets fire at once. A thundering herd on a fixed period.

They are now prime milliseconds near the same tiers: 251 / 499 / 997 / 1999 ms. Mutually coprime, so the LCM is enormous and the buckets' fire times essentially never coincide. Periodical cicadas emerge on 13- and 17-year cycles for the same reason.

This one is honest insurance rather than a measured win — the buckets already run on separate threads, and the global governor scales all four uniformly so coprimality survives throttling. It cost two lines and it future-proofs more buckets, bigger maps, and any shared resource added later.

Identity: the bug that shaped the design

The first implementation keyed plans by the monster's array index, with a generation counter bumped on level reload to catch stale ones.

An index is not an identity. Any mid-flight removal or reorder of the monster list shifts every index above it — and the editor's erase brush does exactly that, while the world is simulating, without touching the generation counter. An in-flight plan would then be applied to whichever monster slid into the vacated slot. Death, level teleport, and a future IQ-driven bucket change are all the same hazard. Bumping the generation at each site is a bandaid, and a footgun for whoever adds the next site.

Every monster now carries a stable, session-global, monotonic runtimeId, assigned once at creation and never reused. Snapshots and plans carry it; the main thread resolves it by lookup when consuming a plan. A plan whose monster has died, been erased, changed bucket, or vanished on a level change simply finds no match and is dropped. It cannot be misapplied.

The generation counter then deleted itself: globally unique ids mean a stale plan never matches a live monster, so it self-invalidates. Fewer moving parts, and no rule to remember.

This is the general rule the rest of the codebase follows — worker ids, entity ids in save files, GPU descriptor slots. Identity is never a position in an array. Positions are an implementation detail of whatever container happens to hold the thing today.

Generalising: an engine-level thread manager

The AI could have owned its four threads directly. Instead they were pushed down into Core::ThreadManager — a registry of named, long-lived workers — and the AI became its first client.

The model: a worker runs a job once per tick, in a loop the manager owns. The job does one unit of work and returns. The manager owns the loop, the cadence, cancellation, timing, and crash capture.

That inversion is what makes the rest possible, because it is now uniform across the whole engine rather than bespoke to the AI:

• Cooperative cancellation. Every job receives a std::stop_token and must check it. The manager's interruptible sleep wakes the instant a stop is requested.
• Non-blocking inspection. Per-worker statistics are atomics, so reading a worker's state — heartbeat, iteration count, tick timings, last error — never blocks it.
• Watchdog and supervision. A tick running past its budget is reported as Stalled (a derived view; the worker keeps running). Past a grace window, an opted-in worker is rebooted.
• Throttling. Pause, live rate change, and a global governor that scales every worker's cadence at once.
• Quarantine. A worker that will not stop cooperatively can be force-terminated, which poisons its slot until an explicit restart.

The AI workers run with a 100 ms watchdog and auto-restart enabled.

What it cost

Snapshot construction, every frame, forever. Publishing is not free — it is the price of the purity that makes the threading safe. Three mitigations bring it near zero:

• Static walkability is shared, not copied. It rides a shared_ptr<const vector<u8>> rebuilt only when the map's revision changes, so a publish copies a pointer.
• Dynamic sets are flat grids, not node containers. Blocked cells and per-cell occupancy are flat mapW × mapH byte arrays indexed z * mapW + x. A set or unordered_set frees its nodes on clear() and re-allocates on every insert; a flat grid is zero-filled in place.
• Snapshots are pooled. The publisher reuses a buffer whose only remaining reference is the pool's (use_count() == 1 — not published, not held by any worker). Plan batches are pooled per bucket the same way. Steady-state publishing allocates nothing.

Latency, deliberately. A plan is stale by up to one bucket interval. This is the entire point — but it means anything requiring an instant reaction cannot live in THINK. Kite and Flee therefore carry no path: the main-thread executors drive them directly from the live party position.

Cancellability is a real constraint on the algorithm. A worst-case full-map BFS must poll its stop token, or a stop request cannot be honoured within the grace window and the supervisor falls through to TerminateThread. TerminateThread runs no unwinding — a worker killed mid-malloc leaks the CRT heap lock and deadlocks the entire process on the next allocation. The BFS polls; ComputeBucket checks between monsters. A heavy tick abandons within one search.

Concurrency you cannot see is concurrency you cannot debug. Hence the dev console panel — which was not optional.

What it bought

• Pathfinding cost leaves the frame. Re-planning is paid on another core; the frame pays a snapshot publish and a plan drain.
• Cost scales with monster count, not with search difficulty. A pathological map costs a worker a long tick, not a dropped frame.
• Reaction time became a tuning knob — a data-driven one, on a stat that already existed.
• The infrastructure is reusable. Asset streaming and any future background work inherit cancellation, inspection, throttling, and supervision for the cost of one Spawn call.
• The failure modes are visible and survivable rather than a hang.

How it's proved

Two mechanisms, and neither is a unit test.

The dev console THREADS panel. Every worker, live: state, iteration count, last / average / worst tick duration, heartbeat age, effective rate against nominal, restart count, last error. Plus commands to pause, throttle, kill, restart, and reap. The re-think period is shown in milliseconds alongside the rate, because "3.98 Hz" and "251 ms" answer different questions.

tools/ThreadStress — a harness driving the real system. It uses the actual Core::ThreadManager and the actual ai::AsyncDirector, publishing snapshots exactly as the game does. Only the world is synthetic: fabricated maps with a controlled monster count per bucket and a controlled search cost, so each bucket can be loaded independently.

Its most valuable phase deliberately drives bucket 0 past the 500 ms force-terminate line to exercise the supervised reboot, and asserts the worker comes back cleanly — restart counter climbs, never quarantined, no force-terminate warning in the log. Before the BFS polled its stop token, that path risked the heap-lock deadlock. The harness is the only thing that ever proved it does not.

Still open

• Plan consumption resolves the monster by linear scan over the live list. Trivial at these counts; it wants an id→index map if monster counts ever grow by an order of magnitude.
• IQ thresholds (130 / 100 / 70) are deliberately coarse placeholders, to be tuned against the catalog's iq values once there are enough monster types to tune against.
• Nothing yet re-buckets a monster whose IQ changes at runtime. The stable-id design already tolerates it — a plan from the old bucket is simply dropped — but nothing triggers it.