The ReAct Architecture

Detailed Study Notes

A customer writes in: "My account C1002 shows past due — I already paid." A plain chatbot answers from thin air: "Your payment is being processed, thank you." It has no idea what C1002's balance actually is. It cannot look. It cannot check whether a refund was filed. It just pattern-matches a soothing sentence and hopes. The gap between that and a real support agent is the whole subject of this note: how do you get a model to go find out before it answers, and to keep going until it actually knows?

Why a model needs a scratchpad

A language model is a stateless next-word predictor — it has no place to "think" except the text it writes. Transformer-based models process a sequence and predict the next token, each token conditioned on the tokens before it in the context window. There is no hidden working-memory register, no notepad tucked away behind the weights. Once you accept that, one consequence follows hard: if you want the model to reason through a multi-step problem, the only mechanism available is to have it write its reasoning out as tokens, in the context, before it acts.

The name says it: ReAct = Reason + Act. It was formalized in a 2022 paper by Yao et al. at Princeton and Google — "ReAct: Synergizing Reasoning and Acting in Language Models." The finding was that models which interleave reasoning traces with tool calls substantially outperform both pure-reasoning models (that think but can't check anything) and pure-tool-using models (that act without a plan) — on complex, multi-step tasks. That last clause matters, and we'll return to it.

The core loop: Thought, Action, Observation

ReAct runs one repeating cycle — think, act, look at the result — and feeds each result back in as the starting point for the next thought. Three phases, in order, over and over:

Notice the asymmetry in the Action phase. The format of the action is deterministic: the model must emit valid JSON matching the tool schema. But which tool it chooses is non-deterministic — that decision comes entirely from the reasoning it accumulated in the Thought phase. If the thinking is poor, the action will be poor. Reasoning quality upstream sets tool-choice quality downstream.

The Observation phase is deliberately dumb. The tool's output arrives raw and uninterpreted — the agent does not immediately react to it. It loops back into a fresh Thought, where it decides what the observation means, whether it's enough to answer, and what to do next. That extra step is a safeguard: it stops the agent from short-circuiting to a conclusion off a single data point.

When ReAct earns its keep — and when it doesn't

ReAct is not "better." It's a tool you reach for when the problem genuinely needs multi-step reasoning over multiple tools — and a liability when it doesn't. Match the mechanism to the problem.

From a straight line to a cycle

The structural change ReAct forces is turning a one-way pipeline into a loop the agent can walk more than once. The classification graph built before this lesson was a DAG — a directed acyclic graph, a "no-going-back flowchart." Nodes wired by deterministic edges, compiled once, the path from input to output fixed at design time and immutable at runtime.

A ReAct graph is fundamentally different: it contains a cycle. The agent node routes to the tool node, and the tool node routes back to the agent node — which may route to the tool node again. In graph-theory terms you've converted an acyclic graph into a cyclic one: a state machine. LangGraph is built to support this safely, with recursion limits, state accumulation, and cycle detection.

The critical piece is the conditional edge — the "which-way sign" out of the agent. After every agent execution the graph asks one question: did the agent decide to call a tool, or did it produce a final answer? That question, and the routing it triggers, is what implements the ReAct pattern on top of an ordinary graph.

The router: decide continue vs. terminate

The router is a plain Python function that reads the current state and returns the name of the next node. It's the loop's decision point, and it has exactly two outcomes: return the tool node's name (continue the loop) or return a terminal route (end the graph). The implementation is small but load-bearing: inspect the last AI message. If it has a non-empty tool_calls attribute, the agent wants a tool → route to the tool node. If tool_calls is empty or absent, the agent produced a final answer → terminate.

def route_after_agent(state: SupportState) -> str:
    last = state["messages"][-1]
    # non-empty tool_calls  → the agent wants to act
    if getattr(last, "tool_calls", None):
        return "tools"        # continue the loop
    return "respond"          # no tool call → terminate

Because the decision about whether more information is needed is delegated entirely to the LLM, this router stays generic — it works across almost any agent configuration without change.

State accumulates every iteration

The messages field uses the add_messages reducer, which appends rather than overwrites. So across N iterations the message list grows into the full reasoning history: initial user query, then first thought (an AI message, possibly with a tool call), then first observation (a tool message), then second thought, second observation, and so on. When the agent node runs the second time it receives that whole trail as context. There is no special memory API — observations feed back into reasoning simply because they pile up in the context window.

The quiet killer: context window bloating

The most challenging production failure of a ReAct agent isn't a crash — it's a slow rot as the accumulating context degrades the model's own performance. Every iteration appends to the context window, and because that same context is what the model reasons over, a large enough context makes each new thought worse. It gets severe only after many iterations — and by then the context is often too large to recover from without starting over.

The rough arithmetic (order-of-magnitude, per iteration): a thought runs 50–150 tokens, a tool call 30–80, and an observation anywhere from 50 to 500. Five iterations might carry the context from ~650 to ~3,000 tokens on top of the original query — trivial for a modern model. But a complex enterprise ticket (a billing dispute, an account reconciliation) can need 15–20 tool calls, pushing the context past 50,000 tokens. At that scale three problems emerge together.

Strategy 1 — messages modifier (trim)

The simplest fix: hand the model a trimmed view of the history without touching the stored state. A transformation function is applied to the message list just before it's sent to the LLM. The state keeps its full history; only what the model sees shrinks. The common implementation is LangChain's trim_messages utility (also available via the messages_modifier parameter on the newer create_react_agent helper): it takes the full list and returns a subset that fits a token budget, using a rolling window.

from langchain_core.messages import trim_messages

def prep_messages(messages):
    return trim_messages(
        messages,
        max_tokens=8000,
        strategy="last",          # rolling window: keep most recent
        include_system=True,      # system prompt always survives
        token_counter=llm,
    )
# state keeps everything; the LLM only sees the trimmed view

Strategy 2 — summarization node

More sophisticated: insert a dedicated node that compresses the history into a structured summary when it gets too long. The node checks whether the message count or estimated token count exceeds a threshold (say 20 messages ≈ 8,000 tokens). If so, it calls an LLM with a summarization prompt and replaces the bulk of the history with a structured summary containing four things:

It preserves semantic content at the cost of extra LLM calls — more latency and money — but the trade-off is favorable when the alternative is context overflow or steep degradation. In practice the summarizer should run on a cheaper, lower-capability model than the main agent: summarizing is a far simpler task than multi-step reasoning.

Strategy 3 — selective context preservation (best)

The strongest approach: keep a separate structured store of extracted facts, so the raw message history can be trimmed hard without losing anything that matters. Rather than blindly trimming or summarizing, the agent maintains a structured knowledge base alongside the message history. Each observation is passed through, its key facts are extracted, and they land in a dedicated state field — for example a customer_facts dictionary inside a TypedDict state.

from typing import TypedDict, Annotated
from langgraph.graph.message import add_messages

class SupportState(TypedDict):
    messages: Annotated[list, add_messages]   # can be trimmed hard
    customer_facts: dict                       # extracted, preserved
    # e.g. {"balance": 240.00, "billing_status": "past_due",
    #        "customer_id": "C1002"}

Because the important facts from every observation are already extracted into queryable state, the message list can be trimmed aggressively without losing information. The agent is explicitly programmed to decide which fields in each tool response are critical facts to keep and which are noise to discard after the current step. It requires more implementation effort than trimming or summarizing, but it's the only one that guarantees precise factual accuracy across many iterations.

Stopping the runaway: infinite-loop prevention

Because the agent decides when to stop, a confused agent may never stop — so production loops need hard, external bounds. A stuck ReAct agent is genuinely expensive: at, say, 10,000 tokens and a second per iteration, a 100-iteration runaway burns minutes and millions of tokens. The loop typically gets stuck for one of three reasons:

Two levels of protection

Protection sits at two levels. At the infra level is a recursion limit set at graph compile time — a hard ceiling that protects against bugs and situations the application logic can't anticipate. At the application level is an iteration count that enables intelligent, user-friendly degradation: instead of a raw graph exception surfacing as a 500 error, the system generates a helpful message explaining what happened.

Detecting repetitive calls with a set

The most precise loop signal is the agent calling the same tool with the same arguments more than once. If the tool returned valid data the first time, a second identical call returns identical data — pointless. If it returned an error, the same call returns the same error. Either way, an identical repeat is a loop indicator, and here iteration and recursion diverge: iteration counts any new node visit; recursion (in this sense) is the same tool + same args firing again and again. You don't want to wait for the iteration ceiling to catch that — you kill it directly.

The mechanism borrows a data-structures move. A Python set holds only unique elements. Store a fingerprint of every tool call — a hash of the tool name plus its sorted arguments — in a set on the state. Before executing a tool, a guard checks whether that fingerprint is already present. If it is, short-circuit: don't run the tool, return a synthetic tool message noting the call was already made and its result already observed.

import hashlib, json

def fingerprint(name: str, args: dict) -> str:
    payload = name + json.dumps(args, sort_keys=True)   # sorted args
    return hashlib.sha256(payload.encode()).hexdigest()

def guarded_call(state, name, args):
    fp = fingerprint(name, args)
    if fp in state["seen_calls"]:          # set of fingerprints
        return {"role": "tool",
                "content": "This call was already made; "
                           "result already observed."}
    state["seen_calls"].add(fp)
    return run_tool(name, args)

When the agent gives up well: graceful escalation

When the breaker fires, the agent must not return a bare error — it should hand off cleanly to a human. Whether the iteration count tripped or the repetitive-call detector fired, the production requirement is a graceful degradation that both tells the end user what happened and briefs the human who takes over. The escalation message has four fixed parts:

Where a human fits in the loop

Human-in-the-loop (HITL) intervention naturally sits at two phases of the cycle. At the Action phase: require human approval before a consequential tool runs — for instance, an agent that decides a customer is eligible for a refund should pause for a person to approve processing that refund. At the Observation phase: let a human review the returned data before the agent's next thought acts on it.

Where this shows up: the CRM support agent

The lesson's running example is a production-grade CRM support agent in LangGraph. A classifier node sorts each incoming ticket into one of four categories — technical, billing, fraud, general — and a router sends it to the matching handler. The billing and technical handlers route into the agent node, the core ReAct reasoner, which has two tools bound to it:

Two guardrail habits from the tool design are worth carrying: every @tool-decorated function gets a precise docstring stating what it does, when to call it, the input format, and the output format — this is what the model reads to choose tools. And the customer-ID tool never guesses or fabricates an ID: if it's missing from the message, the agent must ask the user. [+] That "never fabricate an identifier" rule generalizes to any tool keyed on a user-supplied value.

Interview & practice questions

Key takeaways

1. ReAct exists because models can't think without writing. Reasoning traces are the scratchpad; the loop is Thought → Action → Observation, repeated.
2. The agent decides when to stop — an empty tool call is the exit. That autonomy is the power and the risk.
3. Only reach for ReAct on complex, multi-tool, self-correcting tasks. On simple lookups it's pure overhead.
4. The conditional edge (router) turns a DAG into a cycle — and it must inspect message shape only, never business logic.
5. Context bloating degrades quality, latency (O(N²)), and cost. Fix it with trimming, summarization, or — best — selective fact preservation in a TypedDict.
6. Bound every loop twice: an infra recursion limit (low) plus an app iteration count (graceful), with set-based fingerprinting to kill repeated calls.
7. Fail gracefully with a hardcoded escalation template — never let the failing model write its own hand-off.