Agentic AI · LangGraph
How LangGraph turns a graph definition into a frozen, thread-safe executable, and how an LLM that can only produce text learns to reach into databases, APIs, and calculators through tools.
A LangGraph app follows one rule: compile once, invoke many. Compilation validates the whole graph and freezes it into an immutable, thread-safe object built at startup. And because the model at its center is a read-only text function, every real action — reading a DB, calling an API, doing exact math — must be delegated to a tool: the LLM only requests the call, the framework executes it.
Section 1 · Notebook Summary — hand-copy this
| gives you | because |
|---|---|
| Testability | test fn in isolation; mock the node |
| Error containment | catch at node boundary → error tool message, not a crash |
| Observability | one place to log / meter / trace every call |
Picture a support agent that has just correctly classified a ticket as "billing." What happens next? In the version we built first: nothing. The model returned the word "billing" and stopped. A real human agent would open the customer's account, check the payment history, pull the invoice, confirm whether they were really charged twice. The model can't do any of that — not because it's dumb, but because it has no hands. This lesson is about two structural facts that shape every serious LangGraph app: the graph is frozen once you build it, and the model at its center can only produce text until you give it tools.
2.1An edge connects one node to the next; the type of edge decides whether that connection is fixed or chosen. Before compilation makes sense, the four edge types need to be clear.
A mandatory, sequential step. No decision — node A always leads to node B.
Dynamic. A router function inspects state and returns which node to go to next.
Declares where execution begins — the first node to run on every invocation.
Marks the final state. Execution stops and the accumulated state is returned.
Calling builder.compile() validates the entire graph definition and produces an immutable,
executable graph object. Up to this point you've been describing a graph — adding nodes, drawing edges.
Compilation is the moment the description is checked for correctness and then sealed.
Building the graph is sketching a route in pencil — you can erase and redraw. compile() is sending
it to the printer. Once it's printed, the map is fixed: reliable, shareable, and impossible to accidentally
scribble over mid-journey. Every traveler carries the same map but walks their own path.
The validation pass runs several critical checks before it will hand back a graph:
Once those pass, the graph becomes immutable. You cannot add nodes, remove edges, or change routing
logic at runtime. This is deliberate: if behavior is fully fixed by the definition, the graph is predictable,
inspectable, and safe to run in concurrent environments where many invocations execute at once. The compiled graph
is thread-safe and can be shared across invocations — each graph.invoke() spins up its own
isolated state, so parallel requests never step on each other.
There are two ways to place the graph definition. The wrong way puts build_graph().compile() inside
the function that handles each request. The right way defines and compiles the graph once at module
level, then reuses that compiled object across every invocation.
# WRONG — recompiles on every call
def handle_request(ticket: str):
builder = StateGraph(SupportState)
# ... add nodes / edges ...
graph = builder.compile() # ~50-200ms wasted, every request
return graph.invoke({"raw_input": ticket})
# RIGHT — compile once, at module level
builder = StateGraph(SupportState)
# ... add nodes / edges ...
graph = builder.compile() # runs ONCE when the module loads
def handle_request(ticket: str):
return graph.invoke({"raw_input": ticket}) # reuse the frozen graphCompile at module scope, invoke in the handler. The compiled graph is immutable and thread-safe, so
there is zero benefit to rebuilding it per request — only the cost of re-running validation every single time.
Define it once where the module loads; call invoke() inside the request path.
A conditional edge whose router always returns the same node, with no exit condition, traps execution forever. Fix: wire at least one branch to END, and add a retry-count field to state that caps iterations (roughly three) for that router.
Modifying the state dictionary in place (state["key"] = value) instead of returning a new dict
corrupts shared state and breaks the isolation guarantee. Fix: return a new dictionary from the node;
never assign directly into the input state.
The worked example is a support-ticket classifier: it takes a raw ticket, classifies it as technical,
billing, fraud, or general, and routes it to the matching handler. The
model is Gemini 2.5 Flash at temperature 0. [!] The session summary mentions "Gemini 1.5
Flash," but the actual code sets gemini-2.5-flash; the code is authoritative here.
A plain DAG is a recipe — it lists every step in order and assumes each one succeeds. An agent built on LangGraph is a chef: it reads the recipe, tastes as it goes, decides whether to add salt, and loops back when something's off. The typed-dict state is the chef's working memory, nodes are their actions, and conditional edges are their decisions.
The central contract is the state — a typed dictionary that flows through every node (the real thing carried ~17 fields). The classifier node runs a defensive pipeline rather than trusting the model's raw output:
Normalize strips stray spaces and punctuation and lowercases the result; validate checks it against the four
known categories; if it doesn't match, the category falls back to general rather than crashing. The
graph then routes on that category to one of four handler nodes.
The mental model to keep: we stopped writing recipes and started building chefs. Nodes are actions, conditional edges are decisions, and the typed-dict state is the chef's memory. That framing is what separates an agent from a static pipeline in an interview answer.
An LLM is a text-transformation function: tokens in, tokens out. Reasoning, summarizing, coding, answering — all of it is an emergent property of that single token-in/token-out architecture. That architecture has three consequences every engineer has to internalize before shipping.
Knowledge is a frozen snapshot at the training cutoff. It can't know a payment failed this morning or that an API's schema changed yesterday.
No file system, no sockets, no DB connections, no processes. It runs sealed off; calling it is just an HTTP request to a remote server that can't touch your systems.
It can describe, recommend, or generate code for an action — but it cannot execute anything. It reacts to input and emits output, nothing more.
The passivity is a deliberate design choice, not a bug: it's what makes a model safe to run across shared, multi-tenant cloud environments. But it means anything that depends on information outside the training data will be fabricated, hallucinated, or refused — none of which is acceptable in production.
These map onto four hard limits of a tool-free LLM, useful as a debugging checklist when a production agent misbehaves — is the root cause a missing tool, or something else?
| Limit | What it can't do |
|---|---|
| No database access | Can't issue a SQL query, a Mongo find, or read a Redis key. |
| No API access | Can't make an HTTP request — no Stripe, Slack, Salesforce, email. |
| No real-time data | Prices, exchange rates, weather, health metrics, incident status are all invisible. |
| No deterministic execution | It's a probabilistic text generator, not a reliable calculator — exact math and IDs aren't safe. |
A model that says "your refund is $47.83" in its text has invented that number. A model that calls a
calculate_refund tool and reports what the tool returned has executed real code. The two outputs
can look identical — the difference is entirely architectural. In 2023 a legal tool cited six court cases that
didn't exist and the lawyer was sanctioned. Never use LLM text for anything requiring deterministic correctness.
A tool is a Python function the LLM can request to be called, whose result is returned to the LLM as part of its context. That single sentence hides a three-party contract.
A brilliant consultant who can't enter your building can only give advice. Tools are the specialists inside the building — the accounts team, the database admin — that the consultant can phone. The consultant (LLM) decides who to call and what to ask; the specialists do the actual work and report back.
Does the real work — query a DB, call an API, calculate, read a file. Runs in the agent's runtime, with full access to packages, network, and disk.
The model doesn't call the function. It emits a structured request: "call fn X with args
Y." Passive, not executed.
The framework runs the fn, packages the return value as a tool message, appends it to the conversation, and the LLM reasons over the enriched history to write the final reply.
That last step closes the loop that turns a read-only transformer into an actor: request → execute → append result → answer.
2.6@tool, type hints, and the docstringThe @tool decorator (from langchain_core.tools) turns a plain function into
something the LLM can understand. It performs three operations: schema extraction (inspects the
signature and parameter types), docstring parsing (uses the docstring as the tool's description), and
wrapping (returns a structured tool object that carries its own name, schema, and description).
from langchain_core.tools import tool
@tool
def get_customer_details(customer_id: str) -> dict:
"""Retrieve a customer's billing information and purchase history from the CRM.
Call this tool when the user's query involves billing, invoices, payment
status, or subscription management and is actionable.
Args:
customer_id: The unique customer identifier, format 'CXXXX' (e.g. 'C1042').
Returns a dict with billing_status, last_payment_date, and purchase_history.
"""
record = crm.lookup(customer_id) # real DB call, in the agent runtime
# return ONLY the fields the LLM needs — not the whole raw row
return {
"billing_status": record["billing_status"],
"last_payment_date": record["last_payment_date"],
"purchase_history": record["purchase_history"][:3],
}The LLM never sees your Python source. Its entire understanding of a tool comes from two things: the JSON schema (parameters + types) and the docstring (the description). A vague docstring means the model doesn't know when to call the tool or what to pass. Describe the action, not the implementation — "retrieves billing info from the CRM," never "queries the postgres customers table."
A production docstring answers four questions, in the order the model needs them: what does this do
(describe the action), when should it be called (explicit triggering conditions), what does each
parameter mean and what format (with an example like C1042), and what does it return
(so the model doesn't guess).
Every parameter needs an explicit type annotation. An unannotated parameter defaults to string. If the LLM then passes a string where the function expects an int, you get a type error at runtime — silently, inside the tool executor, and the graph crashes.
Return structured data (a dict → serialized JSON), not a pre-formatted string, so specific fields can be extracted downstream. And return only the fields the LLM needs to answer — every field of a customer row is hundreds of tokens of noise stealing context from the model's reasoning. Filter inside the tool.
Defining a tool and binding it are two distinct operations. @tool creates the
object; bind_tools() tells a specific LLM those tools are available. Binding sends no API request —
it returns a new model object that, when invoked, will include the tools' JSON schema in every request.
Under the hood, bind_tools calls each tool's schema method (the schema the decorator built from the
signature and docstring) and attaches it.
llm_with_tools = llm.bind_tools([get_customer_details]) # no request sent; new object
response = llm_with_tools.invoke("Check account C1042")
print(type(response)) # AIMessage
print(response.tool_calls) # [{'name': 'get_customer_details',
# 'args': {'customer_id': 'C1042'},
# 'id': 'call_abc123'}]When a tool-bound LLM decides to call a tool, it doesn't produce normal text — it produces an AI message
whose tool_calls attribute holds a list of dicts, each with three fields:
The tool/function to call.
A dict of arguments matching the tool's schema, constructed by the model.
A unique identifier for this call, used to match the tool message response.
An AI message can carry several shapes, worth recognizing when debugging:
| content | tool_calls | meaning |
|---|---|---|
| text | empty | LLM answered from context; no tool needed. |
| empty / null | one call | Pure tool request — data needed before it can answer. |
| text | one call | "Thinking out loud" — explains, then requests. |
| empty | multiple calls | Several tools needed to satisfy the query. |
Tool calling was standardized by OpenAI and adopted by Google (Gemini), Anthropic (Claude), and others.
LangChain's bind_tools transparently translates between its tool schema and each provider's
function-declaration format, so your code looks identical regardless of the model — one of LangChain's core
value propositions.
ToolNode is LangChain's built-in execution layer that sits between the LLM and the rest of the graph. It intercepts the tool-call request, runs the matching Python function, and packages the result as a tool message. Its existence as a separate component is a deliberate principle: separate reasoning from execution. The LLM node reasons about what to do; the tool node does it. That separation buys three things.
Test the tool function in isolation in a unit test; mock the whole node in integration tests with stubbed results.
A DB timeout or bad arg is caught at the node boundary and turned into an error tool message the LLM can reason about — retry, tell the user, or escalate — instead of crashing the graph.
One centralized place to log which tool ran, with what args, when, with what result, in how many ms — not duplicated inside every function.
ToolNode is instantiated with the same list of @tool objects bound to the LLM — that's what connects
tool names to references. The tool name in the LLM's response must exactly match a tool name in the
node's list.
LangChain tools use the function name as the tool name by default. Rename a function without re-binding, and the LLM keeps requesting the old name — the node finds no match and the call silently fails. And any unhandled exception that isn't contained propagates up through the node, crashes the graph, and the user sees a 500.
Compilation validates the definition and freezes it so nodes can't be added, edges removed, or routing changed at runtime. Immutability makes behavior fully determined by the definition — predictable, inspectable, and safe to share across concurrent invocations, since each invoke gets its own isolated state and none can mutate the shared graph.
Compile once at module level, at application startup, and reuse the object. Compiling inside the per-request handler re-runs full graph validation on every call, wasting roughly 50–200 ms per request for no benefit — the pattern is "compile once, invoke many."
Temporal isolation (knowledge frozen at the training cutoff), environment isolation (no file system, network, or DB access), and deterministic passivity (it can describe or recommend an action but never execute one). Together these mean any query depending on outside or real-time data will be fabricated or refused without a tool.
The model doesn't execute anything. It emits a structured AI message with a tool_calls list
(name, args, id). The ToolNode intercepts it, validates the args, runs the matching Python function, and
appends the return value as a tool message. The LLM then reasons over that enriched history and produces the
final text answer.
@tool function?The LLM never sees the Python source — its whole understanding comes from the JSON schema (params and types) and the docstring (description). A vague or missing docstring means the model can't tell when to call the tool or what arguments to pass, so the docstring is effectively the translation contract between the model and the function.
A dict serializes to structured JSON, so downstream code can extract specific fields cleanly. You filter to only the fields the LLM needs because the return value is injected into the context window — a full raw row is hundreds of tokens of noise that steal budget from the model's reasoning, so the tool should trim it before returning.
bind_tools() do, and what does it NOT do?It returns a new LLM object that will include the tools' JSON schema in every subsequent request. It does
not send any API call at bind time, and it doesn't define the tools — the @tool
decorator already created them. Binding is just the Python-to-JSON handshake that makes the model aware they
exist.
Testability (test the function alone; mock the node), error containment (exceptions are caught at the node boundary and returned as error tool messages instead of crashing the graph), and observability (a single place to log, meter, and trace every tool call rather than instrumenting each function).
Ensure the router isn't unconditionally returning the same node with no exit. Wire at least one branch to END, and add a retry-count field to the state that the router checks, capping iterations at around three so the graph is guaranteed to terminate.
compile() validates and freezes the graph; do it once at module level and invoke many times.bind_tools() sends nothing; ToolNode executes and contains errors so the graph doesn't crash.
Build it · Chapter 09 · Atlas
LangGraph is a state machine. Three primitives: State (a shared dict), Nodes (functions state → partial update), and Edges (who runs next). compile freezes the wiring; invoke walks it, merging each node's output into the running state. Start with the linear core:
engine.py — keep the file open.atlas/engine.py — the linear StateGraph:
END = "__end__"
class StateGraph:
def __init__(self):
self.nodes = {} # name -> fn(state) -> dict
self.edges = {} # name -> next node
self.entry = None
def add_node(self, name, fn):
self.nodes[name] = fn
return self
def add_edge(self, src, dst):
self.edges[src] = dst
return self
def set_entry(self, name):
self.entry = name
return self
def compile(self):
assert self.entry, "no entry point set"
return CompiledGraph(self)
class CompiledGraph:
def __init__(self, g):
self.g = g
def invoke(self, state, trace=False):
state = dict(state)
node = self.g.entry
while node != END:
update = self.g.nodes[node](state) or {}
state.update(update) # merge this node's output in
if trace:
print(f" [{node}] -> {update}")
node = self.g.edges.get(node, END) # follow the edge
return state
if __name__ == "__main__":
def greet(state): return {"msg": f"hello {state['name']}"}
def shout(state): return {"msg": state["msg"].upper() + "!"}
g = StateGraph()
g.add_node("greet", greet).add_node("shout", shout)
g.add_edge("greet", "shout").add_edge("shout", END)
g.set_entry("greet")
print("final:", g.compile().invoke({"name": "atlas"}, trace=True))Run it — you'll see:
[greet] -> {'msg': 'hello atlas'}
[shout] -> {'msg': 'HELLO ATLAS!'}
final: {'name': 'atlas', 'msg': 'HELLO ATLAS!'}
This is one checkpoint of Atlas — a single project built across notes 04–14. Full code, setup, and the map of all checkpoints: Atlas — Build the notes ↗.