Your AI Agent Needs a Circuit Breaker: Bounding Loops, Tool Failures and Runaway Cost

1. Why a failing tool turns into a runaway loop

In a tool-using agent, the model is the control flow. It reads each tool result, decides whether the task is done, and if not, picks the next action. That design is what makes agents useful, and it is also what makes failure handling awkward. An error message is just more text in the context. Nothing in that text forces the model to stop, so a model tuned to be helpful and persistent can read "Request failed: upstream timeout" as an invitation to try again rather than a reason to stop. That is a plausible failure mode, not a measured one, and it is worth reproducing in your own evaluation set. It may change the arguments slightly, so the attempt looks like progress.

The cost of each attempt is not flat. Every iteration sends the conversation so far back to the model: the system prompt, the tool definitions, the user's request and every previous call and result. A loop appends a new call and a new error on each pass. The input grows, so each step tends to cost more than the one before it. A loop does not charge a fixed amount per retry. Its cost compounds.

Then there is the retry logic you already have. Most HTTP clients and SDKs offer retries with exponential backoff, which is sensible for transient network faults. Inside an agent loop, that logic sits underneath the model's own retries and multiplies them. The model decides to call the tool again, and the client retries each of those calls before reporting failure. A tool that is down gets hit far more often than either layer's configuration suggests.

The underlying shift is that the decision to retry has moved from deterministic code into a probabilistic component. You cannot rely on that component to choose to stop, because choosing to continue is the failure mode. Whatever stops the loop has to sit outside the model and must not depend on the model's cooperation.

2. The most expensive agent failures return HTTP 200

A classic microservice circuit breaker counts failures: exceptions, 5xx responses, timeouts. When failures cross a threshold, the breaker opens and stops sending traffic. That works because a failing dependency in a service architecture announces itself.

Agents usually fail more quietly. These are the patterns I would watch for:

• Repeated identical calls. The agent issues the same tool call with the same arguments, over and over. Each call succeeds and returns the same result, which the model misreads or does not trust.
• Successful but useless results. A search comes back empty, or a document does not mention the entity in question. The model keeps reformulating the query.
• Oscillation. The agent alternates between two tools, such as fetching a record and then validating it, and each result sends it back to the other.
• Steps that change nothing. A step completes, and the agent's working state, plan or draft answer is the same afterwards as it was before.

Every one of these is a success from the downstream service's point of view. The search API returned 200, and the database answered quickly. Your service dashboards stay green while the run's token count climbs. An error-counting breaker never trips, because it has no errors to count.

So an agent breaker has to trip on behavioural signals as well as on downstream errors. The behavioural signals are repeated identical calls, no progress between steps, and budget burn. Error counts catch the dependency that is down. Behavioural signals catch the agent that is stuck, and that is usually the more expensive problem.

None of these signals needs the model's help to detect. To catch repeated calls, hash the tool name together with normalised arguments and count how often each hash appears in a run. Normalising means sorting keys, trimming whitespace and applying consistent casing where casing does not change the meaning. To approximate lack of progress, compare the agent's state or draft output between steps and flag the steps where nothing changed. Budget burn is arithmetic. Track tokens or estimated spend per run and compare the rate against a healthy run of the same kind of task. All three are cheap, deterministic checks that the orchestration code can run on every step.

3. Closed, open, half-open, adapted for an agent

The classic breaker has three states: closed, open and half-open. The state machine carries over to agents largely intact. What changes is the set of signals that feed it and the source of the probe calls.

Closed is normal operation. Tool calls pass through to the real tool while the breaker counts signals: errors and timeouts from the tool, plus the behavioural signals above. Open means the breaker short-circuits. A call returns immediately with a structured "unavailable" result, and the tool itself is never invoked. Half-open is the recovery test. After a cool-down, the breaker lets a limited number of probe calls through. If they succeed, the breaker closes. If they fail, it opens again.

In an agent, the trip decision combines several inputs, each with its own threshold:

• consecutive errors or timeouts from the tool
• repeated identical call hashes within a run
• consecutive steps with no progress
• a spend rate well ahead of normal

These signals are not equivalent, and folding them into one failure counter loses information. A tool that keeps timing out is a different problem from an agent that keeps asking the same question, and each calls for a different response.

Half-open needs more care than it does in a microservice. If the probe comes from the same stuck agent, closing the breaker lets that agent resume the loop that tripped it. Better probe sources are a later run, a scheduled health check against the tool, or a synthetic request sent by the orchestration layer. The stuck run should not be the one that tests recovery.

That leads to two scopes. A per-run breaker answers "is this agent behaving badly?" It trips on behavioural signals, applies only to the current run and resets when the run ends. A per-tool breaker shared across runs answers "is this dependency unhealthy?" It trips mainly on errors and timeouts aggregated over many runs, and it resets through half-open probes. One stuck run can lose access to a healthy tool while that tool's shared breaker stays closed for every other run.

4. Placing the breaker around each tool, outside the model

The breaker belongs in the orchestration code, wrapped around each tool's executor. In a TypeScript agent running on Node.js, the executor is the function that receives the model's tool-call request, validates the arguments, invokes the tool and returns the result to the loop. The breaker sits between the request and the invocation. Every call passes through it, whether or not the model is behaving sensibly.

It is tempting to put this rule in the system prompt instead: "If a tool fails three times, stop calling it." That instruction is a suggestion. The model may follow it, may miscount, or may decide the current case is an exception. The instruction also costs tokens on every step and competes with everything else in the prompt. A prompt can explain the policy; it cannot enforce it. The breaker is the enforcement.

The breaker should be per tool rather than one switch for the whole agent, because failures are local. A failing search API should not disable a calculator or a database lookup that works fine. Thresholds also differ by tool. A cheap, idempotent read can tolerate more retries than a slow paid call. Tools with side effects, such as sending an email or creating a ticket, need the strictest thresholds, because a repeated call does damage in its own right.

Remote tools follow the same rule. When an agent calls tools on an MCP server, the breaker belongs on the client side, in the runtime that the agent operator controls. The server sees a series of individually valid requests. It cannot tell that they come from a looping agent, and it has no access to the run's budget or progress. Server-side rate limits protect the server, but they do nothing to protect the agent's run.

The breaker is one ring in a set of nested controls:

flowchart TB
  subgraph OUTER["Wall-clock timeout + human escalation"]
    subgraph RUN["Run-level step limit + token/cost budget"]
      M["Model"] -->|"tool-call request"| E["Tool executor"]
      subgraph B1["Per-tool breaker"]
        T1["Search tool"]
      end
      subgraph B2["Per-tool breaker"]
        T2["Database lookup"]
      end
      subgraph B3["Per-tool breaker"]
        T3["Remote MCP tool"]
      end
      E -->|"invoke"| T1
      E -->|"invoke"| T2
      E -->|"invoke"| T3
      E -->|"result or open-breaker message"| M
    end
  end

My technical focus on agent architecture, tool use and guardrail design leads me to treat the tool executor as the natural enforcement point. It is the one place where every model decision becomes a real action. The arguments are already structured there, and the orchestration code has the final say, not the model. Prompts, frameworks and model providers change, but the executor is code you own.

5. What the agent is told when the breaker is open

When the breaker is open, the agent still needs a response to its tool call. Throwing an exception is the wrong response. Depending on the framework, an unhandled exception either crashes the run or ends up in the context as a raw stack trace that the model will try to interpret. Return a structured tool result instead. It should have the same shape as any other result, with a clear status.

The content should tell the model four things:

• the tool is unavailable
• the condition will not clear during this run
• the tool should not be called again
• where relevant, a retry-after hint that a scheduler or a later run can use

Where possible, the message should also say what the model can do instead. That might be a fallback tool that covers part of the need, a partial answer built from what has already been gathered, or a question for the user. For example: "web_search is unavailable for the rest of this task. Do not call it again. Answer from the documents already retrieved, or tell the user which part you could not verify."

Wording matters. A vague "temporary error, please try again" invites a retry, because that is literally what it asks for. A message that frames the condition as "unavailable for this task" removes the reason to keep trying and points the model at the paths that remain. This is reasoning about how models treat tool results, not a measured result, so test your own wording against your evaluation set.

The message is guidance, so it should not be the only control. When the breaker opens, also block the tool at the orchestration level. You can drop it from the tool list sent on the next step, short-circuit any call that still arrives, or do both. Then a model that ignores the message still cannot get through.

Sometimes no viable path remains, for example when the only tool that can answer the question is down. In that case, end the run deliberately and explain what was done and what could not be completed, or escalate to a human. Either outcome is much better than a step limit killing the run silently and returning nothing.

6. The layers that catch what a breaker misses

The breaker is one layer among several. Agent frameworks typically expose a step or recursion limit on how long a run can go on. Teams add per-run token or cost budgets, wall-clock timeouts, and the client-level retries described earlier. Each layer catches something different, and no single layer catches everything.

Step limits are blunt. They stop a run only after the waste has already happened, and they report "too many steps" without saying why. Budgets cap the damage but do not diagnose it either. The run ends, and you learn that it was expensive. The breaker is the layer that can stop early and give the model a signal it can act on, but it only covers the failures it was designed to detect.

I would combine the layers rather than pick one. Make the token or cost budget the hard backstop that nothing gets past. Make the breaker the early, informative stop that ends most bad runs before the budget comes into play. Keep the step limit and wall-clock timeout as cheap insurance for failure modes you have not anticipated.

7. False trips, shared state and tuning thresholds

Behavioural breakers produce false positives. Some repeated identical calls are legitimate. Polling a job status sends the same request until the job finishes, and pagination can look repetitive if the cursor is not part of the hashed arguments. These cases need per-tool exemptions or progress-aware detection, such as treating a changed response as progress even when the request did not change.

"No progress" is harder to define than it sounds. Diffing the output is cheap but noisy, because a reworded draft counts as change. Tracking explicit state changes needs a structured agent state, which not every agent has. A cheap judge model can assess progress more flexibly, but it is another model call that adds cost and latency and can fail or be wrong in its own right.

Where the breaker state lives is an operational decision. An in-process breaker in a single Node.js instance is simple and fast, but each replica learns about a failing tool on its own and trips at a different time. Shared state in something like Redis gives every replica a consistent view. The price is a new dependency on the hot path, latency on every tool call, and a decision about what the breaker does when Redis is unreachable.

Thresholds start as guesses. Emit every trip as telemetry, including the signal that caused it, the tool, the run and the step count. Then you can tune thresholds from observed runs rather than intuition, and you can see false trips instead of hearing about them from users.

A tripped breaker is also a product decision. The user gets a degraded answer, or no answer. Decide in advance what degraded behaviour is acceptable for each tool, and do it before the first outage forces the decision.

8. Where I would start in an existing agent

If I were adding this to an agent already in production, I would start with a hard per-run token or cost budget and a step limit. Both are cheap to add, both cap the worst case, and neither depends on getting detection logic right.

Next, I would add a per-tool breaker in the executor that trips on errors and on repeated identical calls. Repeated calls are the most valuable behavioural signal and the easiest to detect: one hash and one counter per run.

After that, I would design the open-breaker message and the graceful-exit path, because a breaker that trips into a crash or a silent failure has done only half its job. No-progress detection and shared breaker state can wait until telemetry shows you need them.

The model can take part in the recovery. It can choose a fallback, write a partial answer or ask the user. It cannot be the component that decides to stop.