Discovery Assistant — LLM-powered legal document review
.NET 10 · Anthropic Claude · OpenAI · Postgres + pgvector
A retrieval-augmented question-answering system for eDiscovery: a reviewer asks questions of a legal document corpus and gets answers grounded only in that corpus — every claim cited to its source passage, and refused outright when the documents don't support one.
At a glance
- Citation-grounded RAG pipeline: ingestion, overlapping chunking, batched embeddings, HNSW-indexed semantic search in Postgres + pgvector
- Model-agnostic LLM layer — Anthropic Messages API for generation, OpenAI for embeddings, both behind ports the application owns
- Structured output with schema validation and a typed corrective retry; streaming with time-to-first-token capture
- Token, dollar, and latency telemetry on every model call — failed attempts included
- Refuses rather than guesses: no supporting passages, no answer — even when the model knows the answer from its training data
- Guardrails as tested invariants: a similarity floor that refuses weak matches before any spend, and PII redaction at the model boundary so identifiers never reach a provider
- Matter isolation enforced in the query layer and proven by test: retrieval cannot cross case boundaries
- Continuous evaluation in CI: a pinned golden set scores retrieval, groundedness, and refusal on every build, with ratcheted thresholds that fail on regression — baselines recall@5 0.977, refusal correctness 1.000, groundedness 0.953
- Verified end to end against the real Enron email corpus
- In active development: agentic tool-calling with scoped memory, and a reviewer UI with a full audit view
Overview
Discovery Assistant applies large language models to legal document review — a domain where a confident wrong answer is worse than no answer, and where every conclusion must be traceable to evidence. It draws directly on my four years building evidence systems for a county attorney's office: the same instincts about provenance, auditability, and matter isolation, applied to a new class of technology.
The premise: anyone can call an LLM API. What separates a demo from a production system is the engineering around the model — grounding its answers in retrieved evidence, validating its output against schemas, measuring what every call costs, and making refusal the default whenever support is weak. In live testing, the system was asked about Enron's Raptor special-purpose entities — a subject the underlying model knows thoroughly from training data, but which the ingested corpus never mentions. It declined, with zero citations, and said what was missing. Grounding beat the model's own knowledge.
The solution is nine .NET projects under a strict inward-only dependency rule (the domain core depends on nothing), enforced by architecture tests that fail the build on a stray reference. LLM providers sit behind ports with retry and instrumentation composed as decorators, Postgres + pgvector provides similarity search inside the database, and the whole system runs from one Docker Compose command.
Highlights
Grounded answers: cited or refused
Retrieved passages are presented to the model as the only permitted source of truth. The structured response must name the passages behind each claim; invalid citations are dropped in code, and a question the corpus can't support returns a refusal as a first-class result — not an error.
Model-agnostic LLM integration
Chat and embedding models live behind application-owned interfaces. The Anthropic adapter uses the official SDK with streaming and adaptive thinking; the OpenAI embeddings adapter is deliberate raw HTTP. Swapping vendors is a composition-root decision, invisible to the rest of the system.
Semantic search inside the database
Chunks are stored with 1536-dimensional embeddings in a pgvector column under an HNSW cosine index. EF Core translates the ranking query so it executes next to the index — vectors never travel to the application. Matter scoping is part of the query signature, so cross-case search is unrepresentable.
Structured output with typed retry
JSON schemas are derived from the C# types they deserialize into — one serializer configuration for both directions, so the contract cannot drift. Output is validated on the way out; an invalid response is retried once with the concrete validation error fed back, then fails with evidence attached.
Cost as a first-class metric
Every model call logs tokens, dollars, and latency — including time-to-first-token on streams and the cost of failed attempts. Ingestion embeds in batches and is idempotent per document, so nothing is ever paid for twice.
Tested at every altitude
Fast unit tests over faked providers; HTTP-seam stubs for the wire mappings; self-gating integration tests that skip cleanly when an API key or the database is absent; architecture tests that enforce the dependency rule. Above them, an evaluation harness scores retrieval, groundedness, and refusal against a pinned golden set in CI, with ratcheted thresholds that fail the build on any regression.
Screenshots
Tech
.NET 10 C# ASP.NET Core Anthropic Claude API OpenAI embeddings Postgres pgvector EF Core Docker Compose xUnit Serilog GitHub Actions