Inside the ReAct loop: how agents actually iterate

There’s a line that’s everywhere now: an agent is an LLM, a loop, and enough tokens. Thorsten Ball put it most directly — 400 lines of Go and you have a real coding agent. Ivan Leo takes a similar angle in his Openclawd From Scratch series, rebuilding an OpenClaw-style agent piece by piece in Python.

Both are excellent. Both glide past the loop itself.

This post is about that loop — specifically, the ReAct (Reasoning + Acting) pattern that powers it. What happens inside one iteration. Why the model is stateless even though the agent feels like it remembers everything. How the transcript grows. When the loop exits. And why the harness — the orchestration code around the model — is where the engineering value actually lives.

The pattern, in one paragraph

ReAct came from a 2022 paper by Yao et al. at Princeton and Google. The idea is small: instead of having the model output just an action, have it output its reasoning alongside the action, in the same generation. Then feed both back in next time. That’s it. Everything else — multi-step tool use, agentic behaviour, autonomous workflows — is downstream of this one trick.

Anatomy of one iteration

One note up front: most production systems today don’t actually parse text for Action: prefixes. They use the native tool-calling APIs every provider now supports — Anthropic’s tool_use blocks, OpenAI’s tool_calls, Gemini’s structured outputs. Architecturally identical to what’s below, just cleaner on the wire. I’ll come back to this variant near the end; for now, the Thought / Action / Observation block is the clearest mental model.

A ReAct iteration is three blocks of structured output, in order:

Thought: I need to check the user's calendar before suggesting dates.
Action: search_calendar(week="2026-04-21")
Observation: Free Tue–Thu, busy Mon and Fri.

The runtime parses the model’s response into Thought + Action, executes the action via whatever tool the model named, captures the result as the Observation, and appends all three blocks to a growing transcript. Then it calls the model again with the appended transcript. Loop.

In OpenClaw — and most production harnesses — the iteration is a fixed pipeline: build prompt → call model → parse response → check if it’s a final answer or a tool call → if tool call, execute → append → repeat.

For the economic reading of this same loop — why turning work into an executable Goal → Task → Tool → Evaluation cycle shifts the boundary of the firm — see Agentic Labour in the Firm Under AI series. This post is the mechanical view; that one is the institutional view.

The model is stateless. The transcript is the state.

This is the part most people get wrong on first read. The model has no memory of previous iterations. Every call is a brand-new inference against a transcript that the runtime maintains on disk. Thorsten makes the point clearly: the server is stateless. It only sees what’s in the conversation slice. It’s up to us to maintain that.

One honest caveat: “stateless” is increasingly a modelling choice rather than a hard fact of the APIs. OpenAI’s Responses API and Anthropic’s conversation-state endpoints let you park the transcript on the provider side and reference it by ID. That’s convenient for demos but it moves the compaction, persistence, and failover problems from your runtime into someone else’s — and those problems don’t go away, they just become less inspectable. Every production harness I’ve seen still treats the model as stateless and owns the transcript locally. The mental model in this post follows that convention.

What this means in practice:

The loop doesn’t exist on the model side. It exists in your code. You can swap providers mid-loop — call Claude for iteration 1, GPT-5 for iteration 2, a local Llama for iteration 3 — and nothing breaks, because there’s no in-flight state to lose. OpenClaw’s Model Resolver does exactly this for failover when an API key gets rate-limited.

The transcript IS the agent’s state. If your runtime crashes between iterations, you can resume by reading the JSONL log off disk and continuing from the last appended block. Harnesses in this family typically store transcripts as line-delimited JSON under a per-agent directory — Claude Code does this under ~/.claude/projects/..., and the same pattern shows up in OpenClaw and most serious open-source equivalents.

The transcript bounds memory. Anything that happened in a previous session, or got summarised away during compaction, is gone unless the system prompt or AGENTS.md surfaced it back in.

Watch the transcript grow

The easiest way to internalise “the transcript is the state” is to watch one being built. Step through the iterations — each tool-using iteration appends Thought + Action + Observation to what came before; the final call appends the final response and exits. By iteration 3 the model has seen everything that happened, even though it remembers none of it between calls.

How a real harness implements this

Abstractly, the loop is a few hundred lines. In a production harness — OpenClaw, Pi, Amp — it gets wrapped in everything else that turns “an LLM in a loop” into a system that can run for days without falling over.

OpenClaw’s architecture makes the layering explicit. A central Gateway handles routing and channel adapters; a single Agent Runtime owns the loop and all its supporting machinery; a few resource stores (JSONL transcripts on disk, skill definitions in folders, tools that wrap real-world actions) sit underneath. Hit play to watch a request cross the whole thing end-to-end.

The same loop you saw earlier sits inside the Agent Runtime box. In a real system, though, there are nine other things that have to happen first and last — channel normalisation at the front, gateway routing, context assembly, model failover, lane-queued tool execution, JSONL persistence, and the response path back through the channel adapter to the user.

Or, if you prefer the temporal view:

Three things this view makes obvious that the architecture diagram doesn’t.

Synchronous depth. Solid arrows are calls waiting for a reply, dashed arrows are returns. Gateway is blocked on Runtime, Runtime is blocked on Model. A timeout at any level cascades back up the stack.

Variable loop count. The dashed loop [ReAct] frame around steps 8–11 can run zero times (model returns a response straight from step 7) or a dozen, depending on the task. Every iteration adds T+A+O to the transcript — exactly as the interactive widget above showed.

Persistence ordering. Step 12 (write to JSONL) happens before step 13 (reply). If the process dies between the model’s response and the user seeing it, the next session will still have the correct transcript. Reply-then-persist would be snappier but would risk drift between what the user saw and what the agent remembers — a class of bug that’s especially nasty in agentic systems where future turns reason over past ones.

Termination conditions

The loop has three ways to end.

The model emits a final response. Most ReAct implementations look for a specific action name — final_response, respond_to_user, or in function-calling APIs, the simple absence of any tool_use block. When this fires, the runtime sends the response back through whatever channel originated the request and the loop exits cleanly.

Max iterations is hit. Most harnesses cap iterations — typically in the low tens. Without it, a stuck loop will burn tokens forever. When the limit fires, the runtime usually injects a “you’ve used your budget, summarise what you have” instruction and treats the next response as final. Not every harness takes this view: Pi deliberately doesn’t expose an iteration cap, arguing a well-prompted model will self-terminate and that artificial knobs just paper over prompt problems. Defensible, but the tradeoff is explicit — a stuck loop is now your problem to notice, not the runtime’s.

The context window guard intervenes. Observations from tools — especially full HTML browser snapshots — bloat the transcript fast. When the assembled prompt approaches the model’s context limit, the runtime triggers compaction (summarising the early transcript into a single block) or stops the loop entirely if compaction can’t recover enough room.

The last two are runtime concerns. The model knows nothing about them.

Why the Thoughts do real work

You might assume the Thought blocks are noise — the model muttering before getting to the actual call. They’re not.

The original ReAct paper compared three setups: Action-only, Reasoning-only, and Reasoning+Acting. Reasoning+Acting outperformed both baselines on the multi-step tool-use benchmarks — HotPotQA and FEVER — even though the Thoughts add tokens (and therefore cost) without doing anything externally observable. Letting the model write down its reasoning before committing to an action measurably improves tool selection.

Empirically: strip the Thoughts and accuracy on tasks that require composition — find X, then use X to look up Y, then decide Z — drops sharply. The Thoughts are also where you do most of your debugging. When an agent goes off the rails, the Thought trace usually tells you exactly where.

The cost is real, though. A Thought before every tool call can easily double or triple the output token count per iteration — the model is generating a paragraph of reasoning on top of the tool call itself. Over a ten-iteration session, that compounds into the kind of token bill that makes the finance team ask questions. Most production systems accept this as the price of reliability: a Thought that costs a few hundred extra tokens is cheaper than a wrong tool call that sends the agent down a blind alley for five more iterations.

The input side has its own compounding problem. Every token in your system prompt and tool schema gets re-sent on every iteration. Mainstream harness system prompts have ballooned to thousands of tokens of preamble — personas, formatting rules, safety caveats — before the first real instruction. Pi goes the other direction and fits its entire system prompt and tool definitions under 1000 tokens, on the theory that frontier models have been trained hard enough that they already know what a coding agent is. Whether you agree with the philosophy or not, the arithmetic is real: prompt tokens × iterations × sessions is the number your API bill actually cares about.

If you’ve ever wondered why Claude Code “talks to itself” before running tools, this is why.

Production failure modes

A few I’ve watched in the wild.

Repetitive loops. The model keeps calling the same tool with minor variations of the same input, convinced one more attempt will work. A common one: model can’t find a file, calls list_files() with slightly different paths six times in a row. Mitigations: deduplicate observations before appending, cap iterations, inject a steering message (“you’ve called this tool 3 times with similar args, try a different approach”) when repetition is detected. Most mature harnesses expose the loop as a hook point for exactly this reason — Mario Zechner’s pi-agent-core is a clean reference implementation.

Premature termination. Model returns a final response before it has enough information. Usually a system prompt or tool description problem. The fix is upstream: clearer tool descriptions, better examples in the system prompt, sometimes an explicit “verify before responding” instruction.

Opaque sub-agents. A common escape hatch when a session gets too big is to spin up a sub-agent for context gathering or a side task. The trouble, as Mario puts it, is that a sub-agent inside an agent becomes “a black box within a black box” — you can’t see what went into its loop, you can’t debug its failures, and its summary back to the parent is whatever the sub-agent decided to say. When it works, it’s elegant. When it doesn’t, you’re debugging one agent’s hallucination about another agent’s hallucination. A safer pattern is to run the context-gathering work as a separate, observable session and hand the artefact back — same information, fully legible.

Context explosion. Tool observations are the usual culprit. A full HTML snapshot of a moderately busy web page runs into tens of thousands of tokens; after two browse calls you’ve used your entire context window on observations, with no room for the actual response. The standard mitigation, used by OpenClaw and most browser-tooling stacks (Playwright-MCP, browser-use), is to parse the accessibility tree instead of rendering full HTML — roughly the same information for the model at a fraction of the tokens. Worth knowing the problem starts before the first observation: Mario Zechner points out that popular MCP servers themselves are expensive — Playwright MCP dumps 21 tool definitions at ~13.7k tokens, Chrome DevTools MCP 26 tools at ~18k — “7–9% of your context window gone before you even start working.” Tool definitions compete for context with tool outputs, and a harness that loads everything upfront is already underwater on long sessions.

Function-calling ReAct

The variant most production systems use today doesn’t actually parse text for Action: prefixes. It uses the native tool-calling APIs that every provider now supports — Anthropic’s tool_use blocks, OpenAI’s tool_calls, Gemini’s structured outputs. Above that layer, protocols like MCP and A2A are rapidly becoming the shared plumbing — what HTTP is to browsers, MCP is becoming to agents. The firm-level view of why that layer matters is The Protocol Layer.

Architecturally identical: still a stateless model in a runtime-driven loop. But cleaner — no parser to write, the API guarantees the JSON shape, and tool definitions ride along in a structured field rather than being shoved into the system prompt as text. Thorsten’s piece is a clean example in Go; Ivan’s series shows the same pattern in Python with Gemini.

The Thought-equivalent is whatever text content the model emits alongside the tool call. It plays the same role.

The harness is the moat

Spend enough time around production agents and one thing becomes hard to ignore: the model is becoming a commodity. You can swap GPT-5 for Claude for Gemini for a local Llama and your agent still mostly works. What doesn’t swap easily is everything around it — the loop control, the tool definitions, the memory architecture, the failure handling, the observability, the security boundaries.

The firm-level version of this argument — that the defensible layer sits above commodity infrastructure, in proprietary protocols, skills, and orchestration — is The Agentic Ecosystem. The harness is the per-runtime instance of that same pattern.

That’s what OpenClaw is. It’s what Pi is. It’s what Amp is. It’s what every serious agent product becomes once it leaves the demo phase. Simon Willison has been making this argument for a while — his framing is that “a coding agent is a piece of software that acts as a harness for an LLM”. The model is the brain; the harness is the body. Almost all the engineering value lives in the body.

The flip side, worth saying out loud: the harness is also where value gets destroyed. Mario Zechner’s argument for Pi is essentially that most harnesses have accumulated enough baggage that they work against the operator — injecting system-prompt boilerplate, bloating tool definitions, burying context decisions. The good harnesses are the ones that stay legible: you can read what went into the model on any given iteration, and change it. Opaque harnesses concentrate value in the vendor; legible ones concentrate it in the operator. Both are defensible business models. Only one is a good engineering choice if you’re the one on call when things break.

The ReAct loop is the canonical pattern that harness implements. It is also where you’d put memory compaction, retry logic, observability, sandboxing, multi-agent routing, skill loading, and everything else. The loop is small. The harness around it is not.

If you want to go deeper, the two best entry points are still Thorsten Ball’s post for the LLM-plus-loop minimalism, and Ivan Leo’s Openclawd From Scratch series for the same idea built up to a real OpenClaw-style agent. Read them in that order, then go read the actual ReAct paper. The loop is small. What you do around it isn’t.