// on air

NeuralGist Radio

Two hosts, one AI idea at a time. Theo and Maya talk through one idea from the site at a time, in about seven minutes.

Episodes

• Boring on Purpose: Traces, Workflows, and the MCP Handshake

  Sep 21, 2026 · 9:44

  Theo and Maya connect three unglamorous pieces of the modern AI stack — LLM observability, workflow-versus-agent design, and the Model Context Protocol — around one idea: you can't ship what you can't see, bound, or reuse.

  Based on: LLM Observability: If You Didn't Trace It, It Didn't Happen · Workflows vs. Agents: Pick the Boring One First · MCP Servers, Explained Like You're Going to Build One

  Transcript

  Theo: Your agent did something strange in production last night. You open the logs and all you have is: POST slash v1 slash messages, two hundred okay, four point two seconds.

  Maya: Congratulations, you know absolutely nothing. Welcome to NeuralGist Radio — I'm Maya.

  Theo: And I'm Theo. We've got three pieces on the site this week that look unrelated. Observability, workflows versus agents, and MCP servers.

  Maya: But they're all the same argument in different clothes, right? The thread is: make the non-deterministic part small and make everything around it legible.

  Theo: That's exactly it. Boring on purpose. Let's start with the logging one, because it's the least glamorous and the one people skip.

  Maya: So what does a real trace actually contain? Because everyone thinks they're logging and they're logging a status code.

  Theo: Per model call you want the full request — the system prompt or at least a hash and version of it, the messages, the tool definitions, the parameters. Then the full response, the tool calls, stop reason, token counts including cached tokens.

  Maya: And cost, computed at the price that was in effect at the time, which I thought was a nice detail. Prices move. Your old traces shouldn't silently re-price themselves.

  Theo: Right. Plus timing — time to first token separately from total latency, because those are different problems. And the part people really skip: trace context.

  Maya: Say more, because I think that's the one that matters.

  Theo: One trace ID for the whole user request, a span for every call inside it. Otherwise a twelve-turn agent run shows up in your dashboard as twelve unrelated rows and you're doing archaeology with timestamps.

  Maya: And the tree view is what actually answers the question people ask, which is never "what was the latency." It's "why did it say that."

  Theo: The example in the piece is a support agent. Turn one, twenty-one hundred tokens in, calls search knowledge base. Tool runs, five results. Turn two, calls get order. Turn three, ends. Whole thing, one tree, one cost number at the top.

  Maya: So you can look at that and go, oh, the knowledge base returned garbage on step two, and every bad thing downstream follows from that.

  Theo: In seconds, instead of guessing. And the dashboards you'd actually build off that data are a short list. Cost by feature by day — that's the one that saves you from the surprise bill.

  Maya: Latency percentiles by model, error and refusal rate, and then a quality signal. That last one is the squishy one.

  Theo: Thumbs up and down, regenerate rate, task completion, eval scores on sampled traffic. None of those are perfect on their own.

  Maya: Regenerate rate is my favorite because nobody has to opt in. If a user hits retry, something was wrong and they told you without filling in a form.

  Theo: There's a volume problem, though. This is a lot of data. Sample if you have to — but keep everything that errored or that a user flagged.

  Maya: Which flows straight into the bit I liked most: evals are observability. Any production trace can become a test case.

  Theo: Bad output, save the inputs, drop it in the eval set, fix it, prove it's fixed. Your trace store is a regression suite that nobody has written yet.

  Maya: That reframing is worth the whole article for me. You stop thinking of logs as forensics and start thinking of them as test material.

  Theo: So that's seeing. The second piece is about shrinking what you have to see in the first place.

  Maya: Workflows versus agents. And the framing is nice and blunt: in a workflow you decide the steps in code and the model fills in each one. Deterministic control flow, non-deterministic content.

  Theo: In an agent, the model decides the steps. You hand it tools and a goal and it loops until it thinks it's done.

  Maya: And the honest line is that agents demo better and workflows are what most shipped systems actually are.

  Theo: The patterns are all things you already know from ordinary engineering. Chain — output of one step into the next. Router — classify the input, dispatch to a specialized prompt. Parallel — fan out ten summaries, fan them back in.

  Maya: Then evaluator-optimizer, which is generate, critique, revise, up to some fixed number of times. Writing tasks love that one.

  Theo: And orchestrator-workers, where one call plans subtasks and workers execute. That's the bridge to agents, and it's where the cost bound starts to get fuzzy.

  Maya: The support ticket example makes it concrete. Classify the ticket. If it's fraud, escalate — no model call at all. Otherwise extract facts, draft a reply, ask a model whether the reply resolves it, and revise once if not.

  Theo: Five calls maximum. Every path visible, every step unit-testable with a fixed input. You know your bill before you ship.

  Maya: So when do you actually reach for an agent? My read: when the number of steps genuinely isn't knowable up front.

  Theo: Debugging, research, open-ended coding. Where the path depends heavily on what you find halfway through, and the task can tolerate variance in cost and time.

  Maya: The test I'll steal: if you can write the steps on a whiteboard, it's a workflow. Build that.

  Theo: And the graduation path is the good part. When you're adding the fifth if-branch to handle a case the model could have figured out, extract that branch into a small agent with a tight budget.

  Maya: Agents inside workflows. Not workflows inside agents.

  Theo: There's also a failure-mode note I want to flag, because it's the thing that bites people. These runs take minutes and touch external systems. They will crash halfway.

  Maya: And if you restart from the top you re-run step one and two.

  Theo: And double-charge someone. So durable execution — Temporal, Inngest, Step Functions, or honestly just a job table with idempotent steps. Resume from step three.

  Maya: Okay, third piece: MCP. And this one is about the tools themselves, which is the part of the trace that isn't the model.

  Theo: Before the Model Context Protocol, every app wrote its own glue for every integration. Want GitHub? Write a wrapper. Slack? Another one. Switch clients, rewrite everything.

  Maya: The protocol just says: integrations are servers, AI apps are clients. Three primitives. Tools are functions the model can call, with a name, a description, and a JSON schema.

  Theo: Resources are things it can read, addressed by URI. And prompts are reusable templates the server offers — "summarize this pull request" as a first-class object rather than something pasted into every app.

  Maya: Transport is either stdio, where the client spawns the server as a subprocess, or streamable HTTP for shared and remote servers. Same protocol underneath.

  Theo: What I appreciate is the article saying plainly that it won not because the protocol is clever. It's deliberately boring. It turned an N-times-M integration problem into N-plus-M.

  Maya: And the mistakes section is where it gets useful for anyone building next week. Number one: tool descriptions are prompts.

  Theo: They're the only thing the model sees. "Search notes" tells it nothing. "Titles of notes whose body mentions the query, case-insensitive substring match" tells it when not to use the tool.

  Maya: Then too many tools — a server exposing eighty of them just eats your context window. Group them, or let clients load definitions lazily.

  Theo: And the security one. Never trust tool output. A resource can contain text that reads like instructions — ignore previous rules and so on. Everything a tool returns is data, not commands.

  Maya: Plus have an auth story before you ship a remote server. Don't build something that will cheerfully delete production because a model asked nicely.

  Theo: So pull the thread. MCP makes your tools uniform, workflows bound where the model gets to improvise, and tracing lets you see both.

  Maya: And each one is the boring choice. Nobody's demoing a router with five model calls.

  Theo: No. But that's the thing that's still running in six months.

  Maya: If I had to give someone one move for Monday — instrument first. You can't decide between a workflow and an agent if you can't see what your current thing costs or where it fails.

  Theo: Agreed. Start with OpenTelemetry and the GenAI conventions as a neutral base, add an LLM-specific layer if you have real users. Roll your own only for a weekend project.

  Maya: That's our show. All three pieces — observability, workflows versus agents, and the MCP walkthrough — are up at neuralgist dot A I.

  Theo: Go trace something. If you didn't trace it, it didn't happen.