5 min read
Multi-agent systems: when it helps
Most multi-agent demos don't beat one well-prompted agent. When coordination is worth the added latency and failure surface, and when it's overhead.
#llm
17 posts tagged llm.
5 min read
Prompt injection: defenses that work
Chunk-level allowlisting, output filtering, privilege separation for agents. Concrete defenses against retrieved-content injection, not the theory.
4 min read
Why eval sets go stale
An eval set built in March measures something else by June. The three things that drift — corpus, query distribution, judge — and how to catch each one.
5 min read
Prompt versioning: stop editing in a CMS
A prompt is an interface. Treat it like code — versioned, eval-gated, with a rollback path — not a string anyone can edit live in a CMS with no review.
7 min read
RAG evaluation: the harness I actually run
Most RAG evaluation advice stops at build a labelled set and use LLM-as-judge. This is the workflow: sampling real queries, and calibrating the judge.
5 min read
Re-ranking: the highest-leverage 50 lines
Reranking top-50 down to top-5 routinely beats raw retrieval by 10+ recall points. What a cross-encoder does differently, and the 50 lines that call one.
7 min read
Swap an embedding model without breaking it
Re-embedding the corpus is the easy part. What actually breaks when you change embedding models in production, and the evals I run on either side of the swap.
4 min read
When not to use an agent
Most problems people reach for an agent to solve are workflow problems with extra steps. A decision framework for telling the two apart before you build.
6 min read
What shipping mobile apps taught me about LLM systems
Mobile engineers spent a decade learning to ship software they couldn't take back. LLM teams are relearning the same four constraints, usually the hard way.
6 min read
What to log for an AI agent audit trail
Not legal advice — the five-field logging pattern I'd want on any agent that takes real actions, and why a log you cannot replay is only a transcript.
5 min read
Choosing a model in 2026: not a benchmark
Pick on task shape — latency, accuracy, cost, context length — not leaderboard rank. A model that wins on average isn't better on every one of your queries.
5 min read
Cost-aware LLM engineering
Prompt caching, model routing, and compression actually move an LLM bill. What each one does, what it doesn't, and the dashboard to build before you need it.
5 min read
5 LLM agent failure modes in production
Tool-use loops with no exit, hallucinated arguments, state leakage, over-confident planning, retry storms — five failures I keep seeing, and what stops each.
5 min read
LLM observability: what to log, alert on
Cardinality, sampling, replay and PII handling for an LLM system — what to log unconditionally, what to sample, and what to keep out of your logs entirely.
12 min read
MCP breaking changes and how to migrate
The 2026-07-28 MCP spec ships a stateless core, deprecates Roots, Sampling and Logging, and assumes OAuth 2.0. What breaks, what to migrate, and when.
5 min read
Chunking strategies for RAG, and when each one bites you
Fixed-size, semantic, and structure-aware chunking all look fine in a demo. Here's how each one actually fails once real documents hit your production pipeline.
7 min read
A production RAG checklist (the boring half)
What separates a working RAG demo from a production RAG system isn't the retrieval — it's the evaluation, observability, and failure-mode handling around it.