AI Agent Architecture Patterns: Single Loop, Graph Workflow, Planner-Executor or Multi-Agent?

1. The structure decides how your agent fails

Prompts and model choice get most of the attention, but the architecture settles four things first:

• who chooses the next step (your code or the model)
• where state lives
• which component holds which permissions
• how a failure surfaces

Those choices fix your debugging, testing and auditing story long before anyone tunes a prompt.

They also determine which failures you get. A loop that wanders keeps calling tools because nothing outside the model tells it to stop. A graph that dead-ends receives an input its designers never pictured and has no edge for it. The loop needs limits and scenario evaluations. The graph needs a fallback path. Rewording the instructions does not reliably fix either one.

Keep workflows and agents distinct. In a workflow, code decides the control flow and the LLM fills in individual steps: classify this, extract that, draft a reply. In an agent, the model decides the control flow. Every design below sits somewhere on that spectrum. Moving along it trades predictability for flexibility.

Frameworks don't make this decision for you. LangGraph can express all four shapes: a loop is a graph with a cycle, and a supervisor is a node that routes. Choosing LangGraph is an implementation decision, and the architecture decision comes first.

2. Four shapes, one question: who owns the next step

The names of these patterns matter less than where each one puts control, state and permissions.

Single tool-calling loop (ReAct-style). One model, one context and one tool set. The model reads the latest result and picks the next action until it decides it is done. This is the quickest design to build. It is also the hardest to bound once the tool list and the context grow, because every new tool adds another move the model can make at every step.

flowchart LR
   subgraph sl["Single loop"]
    LM["Model"] -->|tool call| LT["Tools"]
    LT -->|result| LM
  end

Graph workflow. Developers define the states and transitions. The LLM does classification, extraction or drafting inside nodes, and code decides what happens next. The result is predictable and testable. It is also rigid: an input that falls outside the designed paths has nowhere sensible to go.

flowchart LR
  subgraph gw["Graph workflow"]
    G1["Classify (LLM)"] --> G2["Fetch (code)"] --> G3["Draft (LLM)"]
  end

Planner-executor. One call produces a plan, and executors carry it out step by step, with an optional path back to the planner. Because the plan exists before anything runs, you can inspect what the system intends to do, and that is useful. It does not make the system reliable. A plan is a model output like any other. It can be wrong, it can go stale by the time step three runs, or it can name steps that nothing can execute. A plan needs validation and a replanning path.

flowchart LR
  subgraph pe["Planner-executor"]
    PP["Planner"] --> PV["Validate plan"] --> PX["Executor"]
    PX -->|step failed| PP
  end

Multi-agent (supervisor or peer). Separate agents with separate prompts, contexts and permissions hand work to each other, either through a supervisor that routes or directly between peers. Coordination becomes a design problem you own. You have to decide who owns the task at each moment, what happens when a sub-agent fails halfway, and how two partial results get reconciled into one answer.

flowchart LR
  subgraph ma["Multi-agent"]
    MS["Supervisor"] --> MA["Agent A: own context and tools"]
    MS --> MB["Agent B: own context and tools"]
    MA --> MS
    MB --> MS
 end

3. One problem, four designs: a support assistant that can issue refunds

The problem is a support assistant for a multi-tenant SaaS product. It reads a customer ticket and looks up account and billing data for that tenant. It checks the refund policy, issues a refund under a threshold and escalates everything else. Three constraints shape the design. Tenant isolation is non-negotiable. The refund tool is a write action with money attached. Every decision must be auditable.

Tenant isolation and authorization are concerns I keep returning to in multi-tenant architecture. They matter more here, because the component deciding which data to fetch is now a model.

All four designs work against the same state. The shape below is illustrative and has not been run. In production it would be a validated schema, persisted after each step.

from typing import Literal, Optional, TypedDict

class TicketState(TypedDict):
    tenant_id: str                      # from the authenticated session, never from ticket text
    ticket_text: str                    # untrusted input
    category: Optional[str]
    account: Optional[dict]
    policy_decision: Optional[Literal["allow", "deny", "escalate"]]
    refund_idempotency_key: Optional[str]
    reply_draft: Optional[str]
    owner: str                          # the component that currently owns the ticket
    audit_log: list[dict]

Single loop

One agent gets the read tools and the refund tool, and it handles easy tickets well. The ways it fails are predictable. The model may call refund before it checks the policy. The context may fill with months of billing history until the line that matters gets lost. The guardrails cannot live in the prompt. The refund tool itself has to check the threshold, use tenant-scoped credentials and reject any call that has no policy_decision recorded in state.

Graph workflow

The flow is classify → fetch account → policy check in plain code → draft reply → refund node gated by rules, with an edge to human escalation. Auditing is straightforward because every path exists in code. The failure path is the ticket that doesn't fit. "You double-charged me, and the export button is broken" falls through a classifier built for one intent per ticket. Inputs like that need a fallback edge to a human, or a broader node built to handle them.

Planner-executor

The planner proposes four steps: look up the invoice, check the policy, refund, reply. On a bad run it skips the policy check, or it names a lookup_contract tool that doesn't exist. Validate the plan against an allowed-step schema before execution: the policy check must come before the refund, and only registered tools may appear. When a step fails, send the error back to the planner instead of running the remaining steps blindly.

Multi-agent

This design has three agents. A triage supervisor routes the work. A billing agent is the only one with refund permission. A technical agent has read-only product tools. The isolation argument is real here: the agent that reads untrusted ticket text does not hold write access to payments. The new failure paths come straight from distributed systems:

• Handoff context gets lost, so the billing agent asks the customer for details that are already in the ticket.
• Two agents both reply to the customer.
• The billing agent times out after the refund call but before confirming it, and nobody knows whether the money moved.

4. Side by side: what each design costs you

The table shows predictability and flexibility trading against each other. Graphs win on auditability, and loops win on open-ended input. Planner-executor lets you inspect intent but guarantees nothing about whether the plan is correct. Multi-agent gives you isolation, and you pay for it in coordination.

I won't quote cost or latency figures, because they depend on your models, prompts and traffic. What matters is how the costs arise. More model calls and handoffs usually mean more tokens and more round trips, and each handoff tends to resend context. Parallel sub-agents help only when the work is truly independent. Even then, the slowest branch plus the aggregation step sets the floor. If branch B needs branch A's result, you have a sequential pipeline with extra coordination. Running the branches as agents does not make them parallel.

Look at where the multi-agent column is strongest. The rows where multi-agent clearly wins are the isolation rows, not the capability rows.

5. Most production systems are hybrids

The four patterns combine. A graph node can contain a tool-calling loop, a supervisor can route to a graph, and a planner can emit steps for a workflow engine to execute.

For the refund assistant, I would keep the outer graph deterministic: intake, tenant resolution from the authenticated session, the policy check and an approval gate. A bounded loop handles only the investigation step, where the questions really are open-ended ("which invoice is this about, and was it actually charged twice?"). The refund capability sits in a separately permissioned component that the loop cannot reach.

flowchart TD
  IN["Intake"] --> TR["Resolve tenant from auth session"]
  TR --> INV
  subgraph INV["Investigate: bounded loop"]
    IM["Model"] -->|call| IT["Read-only tools, tenant-scoped"]
    IT -->|result| IM
  end
  INV --> POL["Policy check (code)"]
  POL -->|within auto-limit| RF
  POL -->|above limit| HA{"Human approval"}
  POL -->|unclear or out of policy| ESC["Escalate to support team"]
  HA -->|approved| RF
  HA -->|declined| REP
  subgraph RF["Refund service: separate credentials"]
    RX["Issue refund with idempotency key"]
  end
  RF --> REP["Draft and send reply"]

For this kind of system I would start with the graph and add autonomy only where inputs are genuinely open-ended. Each autonomous region needs its own step limits, evaluation set and traces. Adding one creates work you keep doing for as long as the system runs.

6. What breaks after the diagram is finished

Every design above inherits problems that backend engineers already know from distributed systems. Using a model does not remove them, and it often makes them harder to see.

Partial failure. A tool call or handoff can succeed while its confirmation is lost. For write actions such as refunds, generate an idempotency key before the call and store it in state. Treat the payment system as the source of truth. The model's memory of what it did is not a record.

Ownership and termination. Every task needs exactly one owner at a time, step and budget limits, and a defined end state, which includes "gave up". Without these, loops spin, and supervisors and sub-agents pass work back and forth without anyone noticing. How circuit breakers bound agent loops, tool failures and runaway cost covers the stopping side of this in detail.

Observability. Trace every model call, tool call, handoff and plan revision under one correlation ID so you can reconstruct a single ticket end to end. Multi-agent designs multiply the spans you have to stitch together.

Testing. You can test a graph path by path. Loops and planners need scenario-based evaluations that include adversarial, ambiguous and malformed tickets, not only clean happy-path transcripts.

State and recovery. Persist checkpoints so a failed run can resume or be abandoned safely. Decide in advance what the customer and the support team see when the system gives up.

7. A selection framework you can apply this week

Work through these questions in order and stop at the first one that gives a clear answer for the core of your task. The last question then tells you what else to add.

1. Can you write down the steps in advance for most inputs? If yes, use a graph workflow with LLM calls inside nodes. If no, move to the next question.
2. Is the path genuinely open-ended, but the tools few and low-risk? If yes, a single bounded loop fits. If the tools include writes with real consequences, put those behind gates enforced in code or move them out of the loop.
3. Does the task benefit from an explicit, reviewable plan, and can you validate that plan mechanically? If yes, use planner-executor with a replanning path. If you can't validate the plan, don't use this pattern. The planning step would look like a control without acting as one.
4. Do you need isolation of context, permissions or responsibility? The clearest case is separating the handling of untrusted input from write access. Another is separate teams owning separate agents. Only then use multi-agent, and budget for the coordination work.
5. Which parts actually need autonomy? Combine the patterns accordingly. The answer is usually a hybrid.

These answers rest on an assumption. The framework is for backend teams that have to operate, secure and audit what they ship. An exploratory or internal-only tool can take on more autonomy earlier, because a wrong answer costs a retry rather than a refund. In both cases, the framework chooses the architecture. The framework you build it with comes second.

8. Start with the boundary you need, not the pattern you admire

The useful question is not "how many agents?" It is "where must control, context and permission be separated, and who owns each piece?" Before you open a framework, sketch those boundaries for your own use case, as the refund assistant did. Then add autonomy one bounded region at a time. The components are models now, but the lessons from distributed systems still apply.