AI Engineering · Agentic Systems
Why a straight-line pipeline (a DAG) quietly fails as an AI agent, and how LangGraph fixes it with three primitives: state, nodes, and edges.
A DAG runs each step once, forward, and can never loop back — perfect for ETL, fatal for agents. An agent needs to retry, remember, and self-correct, which requires cycles. LangGraph adds cycles to the execution graph and gives you a shared, typed state that every node reads and writes — turning a one-shot script into a decision system.
The session opened by revisiting a project that motivates everything after it: a PR-risk analyzer that decides whether a talent agency should hire or fire a content creator for a brand collaboration. It fuses three parallel data sources, then compiles a verdict with DSPY.
Per-creator numbers: follower count, bot percentage, average sponsorship ROI. The "math" layer.
Semantic chunks of leaked Reddit / Discord "drama," retrieved by meaning, not keyword.
Creator↔brand↔creator ties: collaborations, "beefs," and shared hidden managers.
A rag_engine.py exposes query_sqlite, query_faiss, and
query_neo4j. A ThreadPoolExecutor(max_workers=3) fires all three in
parallel for a given creator (the "parallel RAG strike"). A DSPY signature then plays a
"brutal, data-literate PR risk analyst" and returns strict JSON: a verdict (hire/fire), a
cancel_velocity score 0–10, and a receipts summary.
DSPY optimization uses a heavy teacher model (Llama-3.3-70B) to generate perfect reasoning
traces, then a cheap student model (GPT-4o-mini) runs those traces in production. You compile
once with BootstrapFewShot, save the optimized state to JSON, and never pay the
expensive optimization cost again at serve time.
In the demo the Neo4j query failed with "cannot connect" — the instructor had not started the Docker container holding the graph DB. The agent still produced a verdict from SQLite + FAISS and flagged the graph as down. Lesson: a well-built agent degrades gracefully when one source is unavailable rather than crashing.
The relevance to today: this analyzer is a one-shot forward pass — retrieve, compile, answer. That works because the question is well-defined. The rest of the lesson is about what happens when a question isn't, and a single forward pass is no longer enough.
Copy-ready. Every line here is derivable from the detailed notes below.
| LangChain | LangGraph |
|---|---|
| component library | orchestration layer |
| atomic ops (prompt, LLM, parser, retriever) | coordinates them over time + memory |
| the workers / machines | the factory-floor manager |
categry) =
silent empty-string, 48-hr prod bug| Overwrite | Append |
|---|---|
| replaces value | adds to a list |
| category, retry_count | messages, logs, retrieved docs |
| "one current value" | "accumulate history" |
A support message lands: "My payment was deducted twice but my subscription is still showing as inactive. I need this fixed immediately or I am disputing with my bank." A classic retrieval pipeline does the only thing it knows how to do — it classifies the message once, routes it once, and answers once. But read the message again: it is not one problem, it is three — a billing error, a broken account state, and an escalation threat. There is no single category that fits all three, so the pipeline picks one, answers it, and silently drops the other two.
That failure isn't a bug in the prompt. It's a limit of the shape of the system. This lesson is about recognizing that shape, naming it (a DAG), and replacing it with one that can think in loops.
The "bot answers, you get frustrated, you type CONNECT ME TO AN AGENT, a human takes over" flow in apps like food-delivery support is a stateful agent with conditional routing and a human-in-the-loop escalation edge. You've been on the receiving end of exactly the architecture this lesson builds.
A pipeline is a fixed, linear sequence of steps decided in advance. Early in the course, retrieval pipelines did exactly this: a query flows through a FAISS retriever, into a prompt template, to the model, and back as a structured answer. Deterministic, bounded, and — for that job — effective.
A hammer is a brilliant instrument for driving a nail and a catastrophic one for removing a screw. Pipelines aren't bad; they're a category of tool. The moment a system must decide, retry a failed step, or route based on what a model just said, you've picked up the hammer and you're staring at a screw.
The distinction that matters is when the control flow is decided. A pipeline's route is fixed at build time. An agent chooses its route at runtime, based on what it has seen so far. Everything else in this lesson follows from that one difference.
A DAG (directed acyclic graph) is a pipeline expressed formally, with three mathematical properties. Understanding the properties is what lets you see precisely why it can't behave like an agent.
Every edge has a direction; execution flows source → destination, never in reverse. The developer fixes this at construction time.
No node can reach itself or an ancestor. This guarantees termination — the graph always reaches END.
With no cycles, there's no mechanism for iteration. If step 5 is wrong, you can't loop to step 3 and retry.
Because a DAG can't loop, a bad result at step 5 doesn't stop the machine — it propagates forward until the system either crashes or returns a confidently wrong answer. That silent-wrong-answer mode is more dangerous than a crash, because nothing signals the failure.
Three concrete scenarios show the same limitation from different angles. In each, no prompt engineering can save the architecture — the missing capability doesn't exist in a DAG.
Ask a financial agent: "Is the company's Q4 cash position strong enough to support the proposed acquisition?" It must fetch the balance sheet, fetch the acquisition cost, reason about the two numbers, account for outstanding liabilities, then recommend. If at step 4 it discovers the liability data is incomplete, a DAG has no path back to step 1 to fetch it. It either answers on bad data or fails outright. A production agent must loop back, request the missing data, and re-run the chain.
User: "My account isn't working." Agent: "What error do you see?" User: "Authentication failed when I try to log in." A DAG processes each message in isolation — the second message arrives with no memory of the first, so it doesn't know this is an answer to a clarifying question. It emits a generic "authentication errors" reply, having forgotten it was mid-diagnosis.
A DAG is stateless between invocations — every call is a fresh start. For multi-turn conversation, which is the primary interface for enterprise chatbots, that isn't merely inconvenient; it's an architectural disqualifier.
An agent generates a structured JSON report and the model emits almost-valid JSON with a
trailing comma. In a DAG the parser node hits bad JSON, raises, and the pipeline crashes — the user sees
an error. In LangGraph the parser writes a parse_error field into state; a conditional edge
downstream reads it and routes back to a "fix formatting" node, which hands the malformed JSON back to
the model with instructions to repair it. The parser re-runs, succeeds, and the user sees a slight delay
instead of a failure.
The move that unlocks self-correction: don't raise on a bad output — write the
failure into state as a field and let a conditional edge decide what to do next. Parsing
failures are inevitable whenever an LLM produces structured output, so recovery has to be
part of the graph, not an afterthought.
LangGraph's core innovation is allowing cycles in the execution graph — a path that leads back to an already-visited node. That single capability enables three things a DAG forbids: iteration, feedback, and self-correction.
The most common cycle is the reasoning loop: the model proposes an action, a validator
evaluates it, and the validator either approves the output or — once a max_retries ceiling
is hit — routes to an escalation or fallback node.
A cycle without a ceiling can run forever and blow the token budget. Always bound the loop with a
max_retries counter; when it's reached, route to a fallback instead of retrying again.
Cyclic ≠ infinite.
The most common mistake for engineers new to this ecosystem is conflating the two. They operate at different levels of abstraction, and using one where you need the other is the root cause of most agent failures.
LangChain components are the machines — a CNC mill, a quality inspector, a packaging robot. LangGraph is the factory-floor management system that decides the order of operations, routes products between stations, and triggers rework when a defect is found. The machines are essential, but without the management layer they have no coordination, feedback, or intelligence.
LangGraph doesn't replace LangChain — it uses LangChain components as the contents of its nodes. The LLM call inside a node is still a LangChain operation; LangGraph just governs when it runs and where its output flows. Picture one big "LangGraph" box with small LangChain components wired inside it.
Whether it's a two-node toy or a multi-agent system with sub-graphs and human checkpoints, every LangGraph app is built from exactly three primitives. Master these and you can design any agentic system.
A typed dict holding all data across every node. Like RAM in a running process — every function reads from it and writes back to it.
Python functions (or LLM calls) that accept the state, do one computation, and return an updated slice of state.
Connections between nodes. Standard edges are unconditional (A→B); conditional edges route dynamically at runtime.
Knowing what happens between graph.invoke() and the final state is what makes an agent
debuggable. The loop is the same every time.
END sentinel.You could write this routing as buried if/else in Python — every team does
before adopting LangGraph, and it fails at scale. Declaring edges explicitly lets you visualize
the graph before running, stream and inspect state at any node, and replay a
failed run from the same initial state. In production that's not a luxury — it's an
operational necessity.
State is the most consequential design decision in a LangGraph app. Every node writes to it and every routing decision reads it, so a sloppy schema corrupts the agent's memory in ways that are miserable to debug.
A TypedDict is a formal contract: every key's type and presence is declared
once and inherited by every node. Read a key that isn't in the schema and you get a static type error
before the code runs.
from typing import TypedDict, Literal, Annotated
import operator
class SupportState(TypedDict):
ticket_text: str
# Literal is more specific than str — the checker enforces valid values
category: Literal["technical", "billing", "escalation"]
retry_count: int
parse_error: str
# append reducer: new messages accumulate instead of overwriting
messages: Annotated[list[dict], operator.add]With a plain dict, one engineer writes category while another node writes the typo
categry. Neither raises. The graph completes, the final state is silently missing a
field, and every ticket routes to the default handler on an empty string. Billing tickets get
misrouted for 48 hours before someone traces it. A TypedDict turns that into a key error
on the first test run, fixed in 30 seconds.
Well-designed state follows three principles:
Exactly the fields nodes need to communicate — no more, no less.
Each field as specific as possible: Literal["technical","billing"], not bare
str.
Reading the schema makes it obvious which node writes which field.
Every field has a reducer — the rule for merging a node's returned value into existing state. Pick the wrong one and you corrupt memory.
| Strategy | Behavior | Use for | Why |
|---|---|---|---|
| Overwrite | Replaces the field's value entirely | category, retry_count |
There's only one current value at a time; old value is irrelevant |
| Append | Adds new values to a list | messages, error logs, audit trails, retrieved chunks |
Data must accumulate across turns; overwriting would erase history |
If messages used overwrite, every node would replace the whole conversation with its own
contribution. By the second turn the agent would have no memory of itself — total amnesia.
Conversation history is the canonical case for append.
Append has a production risk: a long-running agent accumulates thousands of messages until the list exceeds the context window and the inference call fails. The fix is memory trimming — before appending, truncate to the most recent N entries. A conservative standard is ~20 messages (10 exchange pairs): enough context to stay coherent, with headroom for the system prompt, retrieved docs, and the current query.
A node is a Python function that follows a strict three-part contract. Honoring it is the difference between nodes that work in isolation and nodes that are composable, testable, and production-safe.
Accepts exactly one argument: the current state. No extra args, no globals, no hidden external state.
Does one computation — an LLM call, DB query, API call, calculation, or validation. It may read any state field.
Returns a new dict with only the keys it changed. LangGraph merges it; untouched keys keep their values.
def classify(state: SupportState) -> dict:
# reads what it needs from state, ignores the rest
text = state["ticket_text"]
category = llm_classify(text) # a LangChain operation inside the node
# returns ONLY the field it owns — not the whole state
return {"category": category}A cardiologist doesn't rewrite the entire patient chart — they add their cardiology findings and leave everything the ER wrote exactly as it was. A node adds its slice and returns only that; the partial-return pattern is what keeps nodes focused and modular.
A standard edge is the simplest connection: after node A returns, execution always proceeds to node B (think logging after every call). A conditional edge is where the intelligence lives — after a node completes, LangGraph calls a router function that reads state and returns the name of the next node.
builder.add_edge("node_a", "node_b") # standard: always A -> B
def route_by_category(state: SupportState) -> str:
# router: returns a NODE NAME, never a state dict
return state["category"] # "technical" | "billing" | "escalation"
builder.add_conditional_edges(
"classify",
route_by_category,
{"technical": "tech_handler",
"billing": "billing_handler",
"escalation":"human_handoff"},
)A router function follows a different contract than a node: it does not return a state dict, and LangGraph does not merge its output. Its only job is to examine the state and declare the name of the next node. Mixing up "returns state" (node) with "returns a node name" (router) is a classic beginner bug.
Because looping isn't a prompting capability — it's an architectural one. A DAG is acyclic by definition, so there is no edge that leads back to a previous node, no matter how the prompt is worded. The ability to iterate has to exist in the graph structure; you gain it by moving to a cyclic framework like LangGraph.
Directed (edges are one-way), acyclic (no path back to a node), and single-pass (each node runs exactly once). The acyclic property is what forbids loops, and single-pass is what prevents iteration — together they make retry, self-correction, and stateful memory impossible.
They compose. LangChain is a component library for atomic operations (prompts, LLM wrappers, parsers, retrievers); LangGraph is an orchestration layer that coordinates those components over time with shared state. LangGraph nodes typically contain LangChain calls — one is the machine, the other is the factory-floor manager.
State (a typed dict that is the shared memory every node reads and writes), nodes (Python functions that take state and return a partial state update), and edges (the wiring — standard edges route unconditionally, conditional edges route via a router function at runtime).
Overwrite when there is only one meaningful current value — a ticket's category or a
retry_count. Append when data must accumulate across steps — conversation
messages, error logs, audit trails, retrieved chunks. Using overwrite on
messages gives the agent amnesia; using append without trimming risks state
explosion.
An append-reduced field (usually messages) grows without bound in a long-running
agent until it overflows the model's context window and the inference call fails. The fix is
memory trimming: before appending, truncate the list to the most recent N entries — a common
default is ~20 messages (10 turns), which stays coherent while leaving room for the system
prompt and query.
A node accepts exactly one argument (the state), performs exactly one computation, and returns a dict containing only the keys it changed. LangGraph merges that partial return into the global state, leaving untouched fields intact — like a specialist adding findings to a chart rather than rewriting it.
A node returns a partial state dict that LangGraph merges into state. A router (used by a conditional edge) returns a string naming the next node, and its return value is never merged into state. Its sole job is to read the current state and decide where execution goes next.
Buried if/else logic is invisible: you can't visualize the flow, inspect state mid-run, or replay a failure without reading every line of code, and every new feature means surgery on production code. Declaring edges explicitly gives you visualization, streaming/inspection, and replay of failed runs — observability that becomes an operational necessity at scale.
max_retries ceiling that routes to a fallback.Build it · Chapter 10 · Atlas
A pipeline always runs the same nodes. An agent decides. Add conditional edges — a router reads the state and picks the next node — and the graph can now loop. The moment cycles are possible, a recursion limit stops being optional.
engine.py with the three additions below, then a cyclic graph runs. The full assembled file is in the checkpoint.engine.py — add conditional edges + a bounded loop:
# Three additions to engine.py — then the graph can branch and loop.
# 1) in StateGraph.__init__:
self.branches = {} # name -> (router_fn, mapping)
# 2) a new method on StateGraph:
def add_conditional_edges(self, src, router, mapping):
"""router(state) returns a key; mapping[key] is the next node."""
self.branches[src] = (router, mapping)
return self
# 3) on CompiledGraph: pick the next node, then bound the walk.
def _next(self, node, state):
if node in self.g.branches:
router, mapping = self.g.branches[node]
return mapping[router(state)] # the router decides
return self.g.edges.get(node, END)
def invoke(self, state, recursion_limit=25, trace=False):
state = dict(state)
node = self.g.entry
for _ in range(recursion_limit): # cycles are now possible
if node == END:
return state
update = self.g.nodes[node](state) or {}
state.update(update)
if trace:
print(f" [{node}] -> {update}")
node = self._next(node, state)
raise RuntimeError(f"recursion limit ({recursion_limit}) hit — cycle?")
# A cyclic graph: loop through one node until a counter reaches 3.
def tick(state): return {"n": state["n"] + 1}
def route(state): return "loop" if state["n"] < 3 else "done"
g = StateGraph().add_node("tick", tick).set_entry("tick")
g.add_conditional_edges("tick", route, {"loop": "tick", "done": END})
print("final:", g.compile().invoke({"n": 0}, trace=True))Run it — you'll see:
[tick] -> {'n': 1}
[tick] -> {'n': 2}
[tick] -> {'n': 3}
final: {'n': 3}
This is one checkpoint of Atlas — a single project built across notes 04–14. Full code, setup, and the map of all checkpoints: Atlas — Build the notes ↗.