TL;DR: Use a single (or chained) LLM call when the task is bounded and the steps are known. Reach for an agent only when the task needs dynamic, data-dependent decisions, real tool use, and an unknown number of steps. Reliability comes from constraining the loop: tight tool schemas, validation after every step, bounded iterations, checkpoints, and an eval harness on full trajectories, not just final answers.
How to approach it. Push back on the premise gently: agents add latency, cost, and failure surface, so the bar to use one should be high. Name the single condition that justifies the loop (dynamic control flow the model must decide at runtime), then spend most of the answer on reliability, because that is where agents actually fail.
A strong answer. A single call (optionally a fixed chain or a workflow you orchestrate in code) is right when you know the steps: extract then summarize, retrieve then answer, classify then route. It is cheaper, faster, and far easier to evaluate and debug. Prefer it.
An agent (a model that chooses tools and decides its own next step in a loop) earns its cost only when the path is not knowable in advance: the number and order of steps depend on intermediate results, the task needs real tools (search, code execution, a database) and must react to their output, and a human would also work iteratively. Coding agents, research agents, and customer-resolution agents fit; "summarize this doc" does not.
Reliability is the hard part, because errors compound across steps. A 90% per-step success rate is 59% over five steps; the loop punishes you geometrically.
- Constrain the action space. Tight, typed tool schemas and an allowlist of actions; the model can only do what you let it.
- Validate every step. Check tool outputs and arguments before proceeding; never feed an unvalidated tool result back as if it were trusted instruction.
- Bound the loop. A hard cap on iterations and a budget, so a confused agent fails fast instead of spinning.
- Make progress observable and resumable. Log each step's reasoning, tool call, and result; checkpoint state so a long task can recover.
- Keep a human gate on irreversible actions (spend, delete, external send).
- Evaluate trajectories, not just outputs. Score whether each step served the goal (an LLM judge plus spot checks), measure task success rate end to end, and track where it goes off the rails.
The defensible stance: start with the simplest thing that works (single call), add structure (a code-orchestrated workflow), and only hand control to the model when the task genuinely requires runtime decisions.
Key takeaways
- Default to a single call or a code-orchestrated workflow; an agent is justified only by runtime, data-dependent branching.
- Errors compound geometrically across steps, so per-step validation matters more than a clever prompt.
- Bound iterations and budget, gate irreversible actions behind a human, and never treat tool output as trusted instruction.
- Evaluate full trajectories, not just final answers; that is where you catch the step that quietly went wrong.
What interviewers probe next.
- "Workflow vs agent?" A workflow is steps you orchestrate in code (deterministic, debuggable); an agent lets the model decide the steps. Prefer workflows until the task needs runtime branching.
- "How do you stop error compounding?" Validate per step, bound iterations, and let the agent verify or self-correct against tool feedback rather than trusting its own narration.
- "Latency?" Each step is a model call plus a tool round-trip; parallelize independent tool calls and cache, but accept agents are slower and price the SLO accordingly.
- "How do you eval it?" A suite of tasks with known outcomes, trajectory scoring, and regression tracking on success rate and cost per task.
Common mistakes.
- Defaulting to an agent for tasks a single structured call handles.
- No iteration cap or budget, so a stuck agent loops indefinitely.
- Evaluating only the final answer, missing where the trajectory went wrong.
- Trusting tool outputs as instructions, opening an injection and error-compounding path.