Planning ahead versus deciding as you go
Committing to a plan first makes a run cheaper, more predictable and easier to check, and it is wrong whenever the plan cannot survive what the first step returns. This lesson is how to tell which case you are in, and the hybrid that usually wins.
TL;DR: Planning first buys predictability, parallelism and a checkpoint before anything runs. Deciding step by step buys adaptability. The failure of planning is a plan built on assumptions the first result invalidates, and the fix is not choosing one but replanning on a trigger: plan, execute, and replan when reality disagrees.
Where you are. You have the base loop and the interleaved-reasoning variation. The other axis is how far ahead the model commits. It is the decision that most changes what a run costs and how you supervise it.
Two shapes
Decide as you go. The model sees the current state and picks one next action. Repeat. Every pattern so far has been this.
Plan first. The model produces the whole sequence up front, then your code executes it.
- look up the order
- check the payment status
- read the refund policy for that region
- decide and draft the response
Then the loop runs those steps rather than asking the model what to do next each time.
What planning buys
Four things, and the third is the one people underestimate.
Fewer model calls. One planning call plus execution, rather than a call per step. On a five-step task that is a large saving, because each avoided call would also have carried the accumulated history.
Predictability. You know before anything runs what will be attempted. Cost and latency become estimable rather than a distribution.
A checkpoint before acting. You can inspect the plan, validate it, reject it, or show it to a person for approval. That is enormously easier than supervising an agent mid-run, and for anything with irreversible steps it is close to essential. An agent that proposes "delete these six records" and waits is a very different risk from one that starts deleting.
Parallelism. Independent steps can run at once. Deciding step by step is inherently serial, because step two is not known until step one returns.
Where it breaks
One failure, and it is decisive: a plan is built on assumptions, and the first result may invalidate them.
The plan above assumes the order exists, the payment record is findable, and a regional policy applies. If step one returns nothing because the id was a customer number, steps two through four are nonsense. A system that executes them anyway is worse than a step-by-step loop, because it is confidently doing the wrong thing quickly.
So planning suits tasks where the shape is knowable even though the values are not. "Look it up, check it, decide" is a stable shape. "Investigate why this customer is unhappy" is not, because what you do second genuinely depends on what you find first.
The hybrid that usually wins
In practice most good systems do both, with an explicit trigger to reconsider.
The trigger is the whole design. Replanning on every step is a step-by-step loop with extra cost; never replanning is brittle. Useful triggers, in order of how often they apply:
- A step failed. The plan assumed it would work.
- A step returned nothing, or something the plan did not anticipate.
- The remaining plan no longer makes sense given what was learned, which is a judgement and worth asking the model for explicitly after a surprising result.
- A step budget was consumed without the goal being met.
Two guardrails worth having: cap the replans, because plan, fail, replan, fail is the ping-pong failure at a higher level and is more expensive per cycle. And keep the failed plan in context, so the next one does not repeat the approach that just failed.
Choosing
Plan first when the shape of the task is stable, when steps can run in parallel, when someone needs to approve before anything happens, or when the task involves irreversible actions.
Decide step by step when what you do next genuinely depends on what you find, when the task is short enough that planning overhead is not worth it, or when you cannot enumerate the actions in advance.
When unsure, start with a plan and add replanning. It fails visibly, at a point you can inspect, rather than wandering.
Do this before moving on
Take a multi-step task you have given an agent and write the plan yourself, before running it, as a numbered list.
Then run it step by step and mark where your plan diverged from what actually happened. If it did not diverge, the task should be planned rather than decided step by step, and you have just found a cheaper design. If it diverged at step one, you have found a task that needs adaptability and you now know why.
Go deeper
- Agent design patterns covers both shapes and where each is used.
- Practice question: How do plan-and-execute agents work? is this lesson as an interview question.
- Practice question: How do you build human-in-the-loop checkpoints? is the approval point above, done properly, and it is where planning earns most of its value.
- Practice question: How does tree-of-thoughts search differ from a linear plan? is the version where the agent explores several plans with backtracking.
- Practice question: How do you keep a long-horizon agent reliable? is where replanning triggers become the central design problem.
Key takeaways
- Planning first buys fewer calls, predictable cost, a checkpoint before acting, and parallelism.
- It breaks when the first result invalidates the plan's assumptions, and executing on anyway is worse than not planning.
- The hybrid wins: plan, execute, replan on an explicit trigger such as a failure or an unanticipated result.
- Cap the replans and keep the failed plan in context, or plan-fail-replan becomes an expensive loop of its own.
Check yourself before an interviewer does. Answer from memory first.
Which advantage of planning matters most for a task with irreversible steps?