What the loop actually carries between steps
Every agent pattern is a variation on one loop, and the interesting part is not the control flow but what accumulates in the message list. Understanding that accumulation explains the cost curve, the drift, and most of what goes wrong later.
TL;DR: The loop is trivial. The state is not. Every iteration appends to a growing message list that is resent in full, so the agent's memory, its cost curve, and its tendency to lose the thread are all the same fact viewed from different angles. Design what accumulates and you have designed the agent.
Where you are. You have decided a loop is justified. What follows is the loop itself, at the level of detail you need to build one that survives production. This first lesson is about state, because every pattern in the following lessons is a different answer to the same question: what should the model be looking at when it decides the next step?
The loop is not the interesting part
Written out, it is four lines: call the model, if it wants a tool run the tool, append the result, repeat until it stops or you stop it.
Every named pattern you will meet is that loop with a different arrangement of what goes into the message list and when. Which means the control flow is not where the engineering is. The engineering is in the accumulation.
What accumulates
After three steps the message list contains, in order: your system prompt and tool definitions, the user's request, the model's first tool call, the result of that call, the second tool call, its result, the third, and so on.
All of it is resent on every iteration. There is no server-side memory; the growing list is the memory.
Three consequences follow, and they are the three things people are surprised by.
Cost is superlinear in steps. Step eight does not cost what step one cost. It carries everything before it, so a run of eight steps can cost several times what eight independent calls would. When someone reports that agents are expensive, this is usually what they have found.
The signal-to-noise ratio falls as the run proceeds. By step eight the original request is a small fraction of the input, surrounded by tool output. The instruction that mattered is now buried in the middle, which is exactly the position models use least reliably. This is why long runs drift, and it is a context problem rather than a reasoning problem.
Tool results dominate. People assume the model's own reasoning fills the context. Usually it is tool output: a database query returning forty rows, a search returning ten passages, an API returning a nested object with fields nobody needed. This is the single biggest lever on agent cost and it is covered properly in module 3.
The three things a step needs
Given that, the useful question is not "what happened" but "what does the model need at this step to choose well". Three things, and only three:
The goal, still legible. If the original request has scrolled into the middle of a long context, restate it near the end.
What has been established. Not the raw transcript. The facts learned so far: the account id is 4471, the plan is enterprise, the payment failed on the third.
What has already been tried. Especially what failed. Without this the agent repeats a call that returned nothing, which is the ping-pong failure and the most common way a run wastes its budget.
Notice that a naive append-everything loop delivers the third badly and the second not at all. It delivers a transcript, from which the model must re-derive the facts every step. That works for short runs and degrades exactly as they lengthen.
The design lever
Which gives the move that separates a working agent from a demo: you control what goes into the list, and you do not have to put the raw result in it.
A database query returning forty rows can be appended as forty rows, or as "found 3 matching orders, ids 4471, 4482, 4490". A search returning ten passages can be appended whole, or reduced to the two that mattered. A failed call can be appended as a stack trace, or as "no order with that id, the id may be wrong".
Every one of those choices trades information against context budget, and making them deliberately is most of what this module teaches. The default, appending everything verbatim, is a choice too, and it is usually the wrong one past step four.
One loop is the unit. The current argument in the field is about what happens when you stack them, which is where this lesson's ideas go next.
Do this before moving on
Take an agent run, yours or one you construct, and print the token count of the message list at every step. Then plot the numbers, even roughly on paper.
Two things to look for: how steeply the curve rises, and what fraction of the total is tool output rather than reasoning. The curve is usually steeper than expected, and tool results usually dominate. Those two observations are what the rest of this module and all of module 4 are about.
Go deeper
- Agents and tool use is the base loop as a reference, if you want it restated compactly.
- Agent design patterns names the variations covered in the next two lessons.
- Context engineering is the discipline this lesson is arguing for, treated seriously.
- Practice question: How do you control agent cost in production? is the superlinear curve as an engineering problem.
- Practice question: How do you manage an agent's context and memory? is where this lesson leads, and module 4 covers it in full.
Key takeaways
- The loop is four lines. The engineering is in what accumulates in the message list between steps.
- Everything is resent every step, so cost is superlinear and the original goal is progressively diluted.
- Tool output, not model reasoning, usually dominates the context.
- A step needs the goal, the facts established, and what has already failed. A raw transcript delivers none of those directly.
Check yourself before an interviewer does. Answer from memory first.
An eight-step run costs far more than eight single calls. Why?