LLM Observability: If You Didn't Trace It, It Didn't Happen
Model calls are non-deterministic, expensive, and slow. The only way to debug, cost-control, and improve them is to record everything. Here's what to log and what to look at.
Articles
14 published · newest first
all Machine LearningDeep LearningModelsGPUsPrompt EngineeringContext EngineeringHarness EngineeringLoop EngineeringSemantic LayerChatbots & AgentsModel SwitchingMCP ServersWorkflowsObservability
Workflows vs. Agents: Pick the Boring One First
Most "agent" use cases are workflows in disguise. Knowing the difference saves money, latency, and a lot of debugging.
MCP Servers, Explained Like You're Going to Build One
The Model Context Protocol standardizes how models discover and call tools. Here's the mental model, the three primitives, and a server in forty lines.
Model Switching and Routing: Use the Smallest Model That Works
One model for everything is the expensive default. Routing tasks to different models by difficulty, cost, and latency is how real systems stay affordable.
Chatbots and Agents: The Difference Is Who's Holding the Tools
A chatbot answers. An agent acts. Building either well comes down to state, tools, and knowing when to hand off to a human.
The Semantic Layer Is What Makes "Chat With Your Data" Not Lie
Text-to-SQL demos fall over the moment two people define "revenue" differently. A semantic layer is where you write down what words mean before the model guesses.
Loop Engineering, or Why Your Agent Goes in Circles
An agent is a while-loop around a model. Designing that loop, its stop conditions, its state, and its failure modes, is its own craft.
What a Harness Actually Does
The model generates tokens. Everything else that makes it useful, safe, and observable lives in the harness. Here's what's in a real one.
Context Engineering: The Window Is the Product
A model only knows what's in its context window right now. Deciding what goes in, in what order, and what gets thrown out is most of the job.
Prompt Engineering featured
Prompt Engineering Is Mostly Specification Writing
The prompts that work in production don't look like magic spells. They look like the spec a good engineer would hand a new teammate.
GPUs for People Who Just Want Their Model to Run
You don't need to write CUDA to make good decisions about GPUs. You need to understand memory, bandwidth, and why batch size is the only knob that matters.
What a "Model" Is, and What the Names on the Menu Mean
Base, instruct, reasoning, small, frontier, open-weight, distilled. A guide to the words on the model picker and what they change for you.