All chapters
  1. Start hereSomething has to decide what runs next
  2. Foundations · 1Sequence, branch and loop: the three primitives of every workflow
  3. Foundations · 2State machines, DAGs, events and agent loops: four shapes of control
  4. Patterns · 3One agent with tools, or several specialized agents
  5. Patterns · 4Supervisor, planner-executor, router, swarm and peers: who talks to whom
  6. Patterns · 5Control plane and data plane: where code decides and where a model may
  7. Mechanics · 6Task queues, fan-out and fan-in, concurrency and rate limits
  8. Mechanics · 7Handoffs: what one agent must give the next
  9. Mechanics · 8Human approvals: pause, wait for a person, resume safely
  10. Production · 9Durable workflows: checkpoints, retries, compensation and cancellation
  11. Production · 10Permissions, budgets and termination: what a run may do, spend, and when it must stop
  12. Production · 11Failure propagation, circuit breakers and fallback routes
  13. Production · 12Trace propagation and outcome evaluation across a workflow
  14. Reference · 13Frameworks: LangGraph, Temporal, AutoGen, CrewAI and the provider SDKs
  15. Reference · 14Testing orchestration: transition coverage and fault injection
  16. Reference · 15Advanced: speculative execution, deadlocks, fairness and evaluating many agents
  17. Reference · 16The insurance lab: one quote workflow, end to end
  18. ReferenceGlossary
  19. AboutAbout this site and its sources

Agent orchestration · the control plane of agentic software

Something has to decide what runs next.

A model call answers one question. A quote for a commercial building needs evidence from three places, rules applied in code, a check before anything is stored, and a person who decides, possibly days later. This site starts from sequence, branch and loop and climbs, one idea at a time, to durable workflows with several agents and a human approval, asking one question at each step: who decides what runs next here, code or a model?

For backend engineers who are new to agents, and for agent builders who are new to production workflows. One running example, and code that runs without a model, a network or an API key.

You will understand
Why one model call cannot finish a piece of work that has dependent steps, a wait for a person and a failure.
You will be able to say
What orchestration is, in one sentence, and which pre-AI system you already know that does the same job.
You will know
The quote workflow that every page uses, and which chapter answers which question.

An email arrives in an insurer's submissions inbox. A broker wants a quote for an office building:

From: broker@brokerage.example
To: submissions@insurer.example
Subject: Quote request SUB-1004

Please quote property cover for a masonry office, built 2001, 20,000 sq ft, sprinklered. The owner says the roof was replaced 9 years ago. Wind zone C. No losses.

The insurer wants software to do the first pass on requests like this one, so that its underwriters spend their time on the hard cases. An underwriter is the person at an insurer who decides whether to accept a risk. The software prepares a recommendation; the underwriter decides.

The first idea most teams have is one call to a capable model: put the email in the prompt, add the insurer's guidance, ask for a recommendation. Follow that call through this one request and it runs out of road four times.

It needs a fact it does not have. The email says the roof is 9 years old. An outside roof-data provider says 22. Nothing in the prompt contains the 22; somebody has to call the provider first and wait for the answer.

It must apply rules exactly. The insurer's guidance says that when the two roof ages differ by more than 5 years, a senior underwriter must settle it. "More than 5" is arithmetic. It belongs in code that gives the same answer every time, not in a sentence the model may read differently on another day.

It cannot wait. After the recommendation is written, nothing may happen until a senior underwriter looks at it, and that may take hours or days. A model call that holds a connection open for two days is not a design.

And it can fail halfway. If the provider times out, something has to decide whether to try again, how often, and what to recommend when the answer never comes.

Each of those four is a question of the same kind: what runs next, and under which condition? That is the subject of this site.

Orchestration in one sentence#

Orchestration is the logic that decides what runs next, under which conditions, and how the pieces coordinate.

Take the four problems above. "Call the provider before drafting" is a decision about order. "Escalate when the ages differ by more than 5 years" is a condition. "Stop and wait for a person" is coordination with someone outside the program. "Try the provider once more, then continue without it" is a decision about failure. None of these is the work itself. The work is looking up a roof, reading guidance, writing a paragraph. Orchestration is the logic around the work.

Three more words are needed before anything else, and each one is small.

A workflow is the written-down definition: the steps, and the rules for which step follows which. For example: "after the three lookups, run the eligibility rules; after the rules, draft the recommendation".

A run is one execution of a workflow for one submission. The request above becomes run run-b031. Tomorrow's request from another broker is another run of the same workflow.

The orchestrator is the program that executes runs. It holds the state of each run, starts each step, waits for it, and applies the conditions that choose the next one. In the example code of this site, the orchestrator is a small engine of about 600 lines of plain Python.

One more word comes up on every page. An agent is a program that calls a language model in a loop: the model asks for a tool (a function the program runs for it, such as a roof lookup), the program runs it and hands back the result, and the model asks for the next one until it answers. To the orchestrator, an agent is one kind of step. It starts the agent, waits, and reads what came back.

Plain English. The agents do the work. The orchestrator decides which work happens when, and what happens when work fails or must wait.

You have built this before#

Orchestration is older than language models. If you have written a batch job, you have written an orchestrator's instructions.

In Spring Batch, a Job is, in the words of its reference documentation, "a container for Step instances" that "combines multiple steps that logically belong together in a flow". Steps run in sequence with next, and conditional flow sends the job somewhere else when a step ends with a given exit status, for example on("FAILED").to(stepC). A JobRepository stores the state of each execution, so that a restarted job can, in the documentation's words, "start from where you left off". The documentation also allows parallel flows with split. Here is a conditional flow, assembled from the examples on that documentation's "Controlling Step Flow" page (documented shape; not run here):

@Bean
public Job job(JobRepository jobRepository, Step stepA, Step stepB, Step stepC) {
    return new JobBuilder("job", jobRepository)
                .start(stepA)
                .on("*").to(stepB)
                .from(stepA).on("FAILED").to(stepC)
                .end()
                .build();
}

Every idea in that snippet returns in the quote workflow. Steps in order. A condition on the result of a step. Saved state that survives a restart. Parallel branches that join again.

For a backend engineer. Read the quote workflow as a batch job whose steps include model calls, or as a saga: a chain of steps in which each finished step that changed the world has an action that undoes it if the whole thing is cancelled. The comparison holds for order, conditions, saved state and compensation. It stops holding in three places. A step backed by a model can return a different result for the same input. In one mode of the example, a model and not the job definition chooses the next step. And one step waits for a person, for days, which a batch job is not built to do.

Those three differences are why the subject deserves a site of its own. The rest of the site keeps the backend vocabulary wherever it still fits, and says so where it stops fitting.

The quote workflow in one minute#

Every page of this site uses one example: the quote workflow of a fictional insurer that sells commercial property insurance through brokers.

An intake step reads the broker's email once and writes the facts as one structured record, called a submission. From then on every step reads the submission and never the email. In the scenario a submission has 63 fields; the example code carries ten, enough for its rules. The submission above is SUB-1004.

Then three lookups run at the same time: a roof assessment from the outside provider, the insurer's written guidance, and similar past decisions. When the results are in, deterministic rules decide what the facts mean. The underwriting agent drafts a recommendation: one of approve, decline, request_info or escalate, with reasons and the ids of the rules it relied on. Code checks the draft before anything is stored. The recommendation is stored once, and the run waits for an underwriter, who reads it in a web app called the underwriter cockpit and decides. The model recommends; a person decides.

The rules come from the policy fixture, an illustrative rule set invented for this series. The one that matters for SUB-1004 is R-ROOF-03: when the roof age on the application and the provider's roof age differ by more than 5 years, escalate to a senior underwriter.

Diagram: The quote workflow · eight steps, and who decides what runs next at each one

Read left to right. Each column is one stage of one run. The orchestrator starts every box, waits for it, and picks the next one by a condition in code. Two boxes hold a model's work (intake and the draft); the rules and the guard are plain code; the last stage waits for a person. Click a box to see it in case UW-004.

  1. 1 · From email to record
    1. intake: The intake agent turns the broker's email into one JSON record, the submission. Every later step reads the record, never the email.
  2. 2 · Evidence, at the same time
    1. roof_assessment: Asks the roof-data provider (a stand-in) for its view of the roof. Without it the run cannot recommend approve or decline.
    2. guidance: Retrieves the rules of the policy fixture that the draft may cite.
    3. similar_decisions: Finds past decisions on similar buildings. If it fails, the run continues and says it is missing.
  3. 3 · Rules, draft, check
    1. check_eligibility: Applies the deterministic rules of the policy fixture. Same input, same result, every time.
    2. draft_recommendation: The underwriting agent writes the recommendation with reasons and the rule ids it relied on.
    3. verify_process: Code checks the process before anything is stored: the rules ran, and every cited rule was really retrieved.
  4. 4 · Store, then wait
    1. record_recommendation: Stores the recommendation for the underwriter cockpit. A key makes a repeated write store nothing new.
    2. await_underwriter: The run stops holding compute and waits for an outside event: a person's decision.
  5. 5 · A person decides
    1. apply_decision: Stores the decision a person made in the underwriter cockpit. Without that signal there is nothing to store.

Read the diagram for who acts. Two boxes hold a model's work: intake and the draft. The rules and the guard are plain code. The write happens once, after the guard. The last box belongs to a person. And between every pair of boxes sits a decision the diagram does not draw: the orchestrator choosing the next step.

The same eight steps as a table, with the answer to the site's question in the last column:

#StepWhat it doesWho decides what runs next
1intakeThe intake agent turns the email into a submissionCode
2gather_evidenceThree lookups at the same time, then joinedCode (pipeline mode) or a model inside a budget (loop mode)
3check_eligibilityRules of the policy fixture, in codeCode
4draft_recommendationThe underwriting agent writes the recommendationCode
5verify_processA guard in code: the rules ran, every cited rule was retrievedCode
6record_recommendationThe one write, safe to repeatCode
7await_underwriterThe run stops and waitsA person
8apply_decisionStores the person's decisionCode, from the person's signal

The middle column is the work. The last column is orchestration. Notice how little of it is a model's choice: in pipeline mode, none. In loop mode, the example replaces steps 2 to 4 with one agent loop in which a model picks the next lookup, inside a budget, and a code guard checks the result at the exit. Steps 5 to 8 stay fixed. That mix, a fixed skeleton with a bounded choice inside one step and a person at the decision, is the design this site recommends; Control plane argues it in full.

One run, start to finish#

The example ships with the site: examples/uw-orchestrator/, standard-library Python. The agents here are scripted stand-ins and time is simulated, so these results show how the orchestration works, not how good or how fast any model is. The command runs case UW-004, the roof conflict above, in pipeline mode. A successful run proves that the orchestrator took the eight steps in order, stored one recommendation, waited for a person and applied that person's decision. It does not prove anything about the quality of a recommendation.

cd examples/uw-orchestrator
/opt/homebrew/bin/python3 run.py --case UW-004 --mode dag
Quote workflow · scripted stand-in agents (no model called) · simulated time · seed 7
UW-004 · Application and provider disagree on the roof age · dag mode · run-b031 · trace ebef10e31f83...
  path:      intake > gather_evidence > check_eligibility > draft_recommendation > verify_process > record_recommendation > await_underwriter > apply_decision
  fan-in:    roof_assessment ok [ok], guidance ok [ok], similar_decisions ok [ok]
  guard:     pass (Eligibility ran and every cited rule was retrieved.)
  recommendation: escalate · cites R-ROOF-03 · missing - · policy 2026.2 · stored row status stored
  signal:    senior_underwriter approve -> apply
  human_decision: {"decided_by": "senior_underwriter", "decision": "approve", "override": false, "reason": "The inspection report on file confirms a roof age of 22 years, which R-ROOF-01 allows."}
  status:    completed · model calls 3 · tool calls 4 · simulated time 3.0 h (includes the wait for a person)
  expected:  as stated in the case
wrote 1 run file(s) in out/runs/ and out/summary.json

Three lines carry the story. The path line is the sequence the orchestrator chose. The signal line is the outside event that ended the wait: a senior underwriter, because the recommendation was escalate, approved. And the status line counts 3.0 simulated hours, almost all of it spent waiting for that person. A single model call would have had to hold on for all of it.

In the quote workflow. This is an illustrative reconstruction for teaching. The insurer, the cases, the rules, the fields, the provider, the timings and every number are invented. Nothing here reports the quality of a real system or model.

The same idea in a coding agent#

Orchestration is not about insurance. A coding agent that fixes an issue in a repository has the same moves, and you may know them from your own tools.

MoveIn the quote workflowIn a coding agent
Steps in orderintake, evidence, rules, draft, guard, writeRead the repository instructions, plan, edit, test, open a pull request
A branchThe rule outcome picks the routeTests pass, or they fail
A loopRetry the roof lookupEdit and test again
Parallel workThree lookups at onceCI jobs in parallel
A person decidesThe underwriter in the cockpitThe reviewer approves the merge
State that survivesCheckpoint after each stepThe branch and the pull request
A budgetTurn limit of the agent loopTurn limit of the agent

The loop inside one coding agent belongs to Harness Engineering; the hooks, review and CI around its change belong to Diff to Deploy. This site is about the logic that strings such steps together.

Try the three moves#

The lab below writes the quote workflow with three moves only: steps in order, a choice on a condition, and a retry loop. Set three facts of a submission (is the roof age known, how old is the roof, which wind zone) and how often the provider times out, then step through the run and watch which branch is taken. The table after it holds five settings with their results, computed by core.py:primitives_walk under policy version 2026.2.

SettingRoof age knownRoof ageWind zoneProvider timeoutsLookup attemptsRouteRule or missing fact
Facts of UW-001yes31C01declineR-ROOF-01
Facts of UW-002yes8A01approveno rule applies
Facts of UW-003noB01request_infoR-ROOF-02, missing roof_age_years
Provider times out onceyes12B12approveno rule applies
Provider times out twiceyes12B22request_infomissing roof_assessment

The last two rows differ by one timeout and end on different routes. Nothing about the building changed. What changed is a decision the orchestration made: the loop allows two attempts, and after the second failure the run takes another route instead of guessing. Primitives builds the whole workflow from these three moves.

Where this fits#

This site is one of a series of guides on building agentic software, and each guide owns one subject. Orchestration touches most of them, so the boundaries matter.

Five neighbors sit closest. When the assessment agent hands its findings to the underwriting agent, this site decides when the handoff happens and what must be in it; the format of the message on the wire belongs to Agent Communication. When the orchestrator saves a checkpoint, this site says when and what for; how saved state and long-lived knowledge are stored belongs to Memory Engineering. When a step fails, this site decides which route the run takes next; delivery guarantees, service levels and incident response belong to Agent Reliability. The trace_id of a run is used here to follow one run across its steps; what a trace and a span are in depth belongs to Observability. And the loop inside one agent, the model asking for tools turn after turn, belongs to Harness Engineering; this site treats that loop as one step and asks who bounds it.

What this is not. Not a guide to messages between agents (see Agent Communication), to storing state (see Memory Engineering) or to operational guarantees (see Agent Reliability). Each page here names the neighbor at the point where the subjects meet and links to it instead of repeating it.

How to read this site#

The chapters follow the order in which the questions come up when a team turns a quote workflow like this one into production software. First the moves and the shapes every workflow is made of. Then the patterns for one agent or several, and where code must keep control. Then the mechanics of parallel work, handoffs and a pause for a person. Then what production adds: crashes, limits, failures, traces. A reference part closes the site.

Diagram: The route through the site · from three moves to a durable run with several agents

Read left to right. Each column is a part of the site and each box a group of chapters. Each part answers the same question for a bigger piece of the quote workflow: who decides what runs next here, code, a model or a person?

  1. 1 · Foundations
    1. Three moves: Sequence, branch and loop, with the words step, condition, state, transition and termination condition.
    2. Four shapes: State machine, DAG, event-driven, agent loop: four ways to write down what runs next, and what each cannot express.
  2. 2 · Patterns
    1. One agent or several: What a second agent adds and costs, and five arrangements of several agents.
    2. Where code decides: Control plane and data plane: a fixed skeleton with a bounded model choice inside one step.
  3. 3 · Mechanics
    1. Parallel work, handoffs, a person: Fan-out and fan-in, what one agent gives the next, and pausing a run for the underwriter.
  4. 4 · Production
    1. Crashes, limits, failures, traces: Checkpoints and retries, budgets and termination, circuit breakers, and one trace per run.
  5. 5 · Reference
    1. Frameworks, tests, edges, the lab: Frameworks read on their own documentation, testing the orchestrator, the hard edges, and the whole example end to end.

Read in order#

Foundations. What every workflow is made of.

Patterns. One agent or several, and who keeps control.

Mechanics. Parallel work, handoffs, a person.

Production. What a real run survives.

Reference. Tools, tests, hard edges, and the whole example.

Jump to a question#

If your question isStart at
Graph, state machine or agent loop: which do I need?Shapes
Do I need a second agent at all?One or many
How much may a model decide about the next step?Control plane
How do I pause for a person and resume exactly once?Human approvals
What happens to a run when the process dies?Durable workflows
How do I stop an agent loop that does not end?Limits
Which framework runs which part of this for me?Frameworks

Checklist#

Before calling a piece of agentic software orchestrated, a reviewer can tick these:

Orchestration and choreographyA central program that decides, or services that react to each other

Distributed systems use two words for coordinating work. In orchestration one program holds the plan and tells each participant what to do next. In choreography there is no such program: each participant reacts to events from the others, and the overall order emerges from those reactions.

The quote workflow is orchestrated on purpose. One place holds the state of each run and decides the next step, so one place can refuse to store a recommendation that skipped the rules, and one place knows that a run is waiting for an underwriter. Choreography spreads that knowledge across handlers, which makes each service simpler and the whole harder to see and to test.

The two mix in practice. The underwriter's decision reaches the run as an event, which is a choreographic move inside an orchestrated workflow. Shapes treats event-driven control as one of four shapes and says what it cannot express on its own.

Why not one long prompt with tools?A single agent can call the provider itself; what it still cannot do

An objection: give one agent all four tools and let it call the provider, read the guidance and apply the rules itself. That removes the first of the four problems on this page, the missing fact. It does not remove the other three.

The rules would then be applied by a model, so "more than 5 years" becomes a judgment instead of a computation, and nothing outside the agent checks that the rules ran at all. Case UW-008 shows the consequence in loop mode: the stand-in model skips the guidance search and still cites R-ROOF-01. In the example a code guard at the exit of the loop catches it and stores nothing; without such a guard the citation would reach the underwriter.

The wait for a person still cannot happen inside the agent, because the agent's loop ends when the model answers. And failure handling inside one agent depends on the model choosing to retry. The example keeps one agent loop as an option, inside one step, with a budget and a guard. One or many and Control plane make the full argument.

How this site uses its wordsWorkflow, run, step, attempt: four words kept apart on every page

Frameworks and papers use these words differently, so this site fixes them. A workflow is the definition; a run is one execution of it for one submission, with a run_id such as run-b031; a step is one unit of work the orchestrator starts and waits for; an attempt is one try of one step inside a run, counted by attempt_number. The business record keeps its own id, submission_id (SUB-1004), which survives across runs. On this site each run has one trace, with a trace_id of 32 hex characters; Tracing explains why the four identifiers must never be mixed up.

The word "task" is kept for an item in a queue and for a framework's own vocabulary. The word "pipeline" names one mode of the example, not orchestration in general. A framework that says "graph", "flow", "crew" or "workflow" may mean something narrower or wider; Frameworks reads each one on its own terms.

In one breath#

A model call answers one question, but a quote needs a fact from a provider, rules applied exactly, a check before anything is stored, a person who decides days later, and a plan for the provider that times out. Orchestration is the logic that decides what runs next, under which conditions, and how the pieces coordinate; the orchestrator is the program that holds each run's state and makes those decisions. It is the batch job and the saga you already know, with three new facts: some steps vary, a model may choose the next step, and a person sits inside the run. So every page of this site asks who decides what runs next here: code, a model, or a person.

You can now say what orchestration is, read the eight steps of the quote workflow, and point at the one place where a model may choose. What you cannot do yet is write a workflow down in a way an engine can run and a reviewer can check. Primitives does that with the three moves every workflow is built from, sequence, branch and loop, and the words that go with them.