Almost every agent tutorial teaches the same pattern: a while (true) that hands the model the conversation and every tool, runs whatever it asks for, feeds the result back, and repeats until the model stops asking. It works in the demo, and it's an honest place to start. The trouble shows up when someone asks "what step is this conversation on?", "why did it call search three times?" or "can we get approval before it publishes?", and the only answer you have is "depends on what the model decides". At Jxbs the agent layer is built with LangGraph around a supervisor node that routes messages by intent, and the lesson that has paid off most is this: the model should decide as little as possible about the flow, and as much as possible about the content.
What the loop doesn't tell you
The free loop looks roughly like this:
while (true) {
const res = await llm.invoke(messages, { tools: ALL_TOOLS });
messages.push(res);
if (!res.tool_calls?.length) break;
for (const call of res.tool_calls) {
messages.push(await execute(call));
}
}
There's nothing wrong with those ten lines. What's missing is everything that surrounds a production system:
- There's no state, only history. All you know about the conversation is a list of messages. Want to know whether the user already confirmed the job post? Re-read the history, or ask the model.
- Cost has no ceiling. The model decides how many turns it takes. You add
maxIterations = 10, and that number doesn't come from reasoning, it comes from fear. - You can't test it in pieces. The only test available is end to end, with a non-deterministic model in the middle.
- There's no room for a human. Pausing halfway, waiting for an approval and resuming hours later doesn't fit a loop that lives in memory inside a request.
State is the contract
A state graph starts with the question the loop skips: what do I need to know about this conversation at every point? In LangGraph that's the state, a typed object every node reads and returns partial updates to:
import { Annotation, StateGraph, START, END } from '@langchain/langgraph';
import type { BaseMessage } from '@langchain/core/messages';
const State = Annotation.Root({
messages: Annotation<BaseMessage[]>({
reducer: (prev, next) => prev.concat(next),
default: () => [],
}),
intent: Annotation<'edit_job' | 'find_candidates' | 'out_of_scope' | null>(),
blocked: Annotation<boolean>(),
draft: Annotation<Record<string, unknown> | null>(),
});
Notice what just happened: intent and blocked are no longer things the model "remembers". They're fields you can read, log and test. The messages reducer says explicitly how updates combine. And when someone asks where a conversation is, the answer is an object, not an interpretation.
Your code picks the edges
The second decision is who chooses the next step. In the loop, always the model. In the graph, edges decide the flow, and most of them can be plain code:
const graph = new StateGraph(State)
.addNode('guardrails', checkInput)
.addNode('supervisor', classifyIntent)
.addNode('editor', editJob)
.addNode('matching', findCandidates)
.addNode('reject', replyOutOfScope)
.addEdge(START, 'guardrails')
.addConditionalEdges('guardrails', (s) => (s.blocked ? 'reject' : 'supervisor'))
.addConditionalEdges('supervisor', (s) => s.intent ?? 'out_of_scope', {
edit_job: 'editor',
find_candidates: 'matching',
out_of_scope: 'reject',
})
.addEdge('editor', END)
.addEdge('matching', END)
.addEdge('reject', END)
.compile();
The supervisor still uses a model, but for something narrow: classify the intent with a structured output whose only possible value is one of three strings. That's a decision you can evaluate against a dataset of real messages and a confusion matrix. "Pick freely among twelve tools in a loop" is not.
And the guardrails node runs before any tool-calling step, every time, because it's an edge and not a suggestion in a prompt. In a loop, the filter running first depends on nobody reshuffling the code; in a graph, it's drawn.
What you get for free
Once the flow is a graph, things show up that you'd otherwise build by hand:
Persistence and resumption. Compile with a checkpointer and every step is saved per thread_id. The conversation survives a server restart, and you can inspect the state of any thread.
const app = graph.compile({ checkpointer: new MemorySaver() }); // PostgresSaver in production
await app.invoke({ messages: [input] }, { configurable: { thread_id: conversationId } });
Pauses for a human. A node can call interrupt() before an action with side effects — publish, send, charge — and the graph stops there with its state saved. When approval arrives, you resume the same thread. That deserves its own post, and it's tomorrow's.
Per-node streaming. You can tell the frontend which step the graph is on, not just stream tokens, which is what makes a streaming chat feel alive instead of stuck.
Per-node tests. classifyIntent is a function that takes a state and returns a state. It's tested on its own, with fixtures, without spinning up the rest.
When the loop is right
Not everything fits a closed graph. Some tasks are genuinely open-ended — researching something, exploring a repo, debugging — where you don't know up front how many steps it will take, and forcing them into boxes is worse than letting the model drive.
The answer isn't picking one or the other. It's putting the loop inside a node: a ReAct-style sub-agent with a small tool set, read-only where possible, and a step limit that this time comes from something you measured. LangGraph exposes that as recursionLimit in the call config. The outer graph still knows where that exploration starts and ends, what it cost and what it returned.
My recommendation
Before you write the first prompt, draw the flow on paper. Every box you can describe without saying "the model decides" is a plain code node; every arrow that depends on an if over the state is a conditional edge with no model in it. What's left — the boxes that genuinely need judgement — is where the LLM goes, with the narrowest output you can give it.
If your drawing comes out as a single box with an arrow looping back to itself, start with the loop. But the moment someone needs to know which step a conversation is on, or to approve something before it happens, you'll know the loop has outgrown you.