It is Tuesday afternoon. Run run-019a of the quote workflow has done everything a machine may do for case UW-902. It gathered the evidence, the rules said the roof is fine under the policy fixture 2026.2, the underwriting agent drafted approve, the guard passed, and the recommendation is stored. The model recommends; a person decides. So the run now waits for an underwriter.
The underwriter is in a meeting, then goes home. On Wednesday morning the insurer publishes a new version of the policy fixture, 2026.3 (an illustrative version this site invented for this case), which lowers the roof limit. At ten o'clock the underwriter opens the underwriter cockpit, reads Tuesday's recommendation, and clicks Approve.
Should the orchestrator apply that click?
The click is real. The person is authorized. The recommendation was correct when it was written. And yet the rules it was based on are no longer the rules in force. Applying the click would record an approval that today's policy does not allow. Refusing it outright would throw away a person's decision without telling them why.
This page is about the part of a workflow where code hands control to a person and later takes it back. Three things must go right: the wait must cost nothing while it lasts, the answer must be applied exactly once, and the answer must still make sense when it arrives. One note before any output: the agents here are scripted stand-ins and time is simulated, so these results show how the orchestration works, not how good or how fast any model is.
1. What a human approval is#
Start with the plain idea. A human approval is a step in the workflow where nothing more may happen until a person says yes, no, or something else. In the quote workflow it is step 7, await_underwriter. The run reaches it after the recommendation is stored. It leaves it only when an underwriter decides.
Plain English. The orchestrator puts the case on the underwriter's desk and stops. It picks the case up again when the underwriter puts a decision on top of it.
Why is this a separate mechanism at all? Because every other step on this site is short. A lookup takes seconds, a rule check takes milliseconds, a model call takes a few seconds. A person takes as long as a person takes: an hour, a weekend, a week of leave. A workflow that treats the person like a slow function call breaks on the first long wait.
In the quote workflow.R-HUMAN-01in the policy fixture says every recommendation needs a decision by an authorized underwriter. The rule is illustrative, but the design follows from it: no run ends with an outcome unless step 8,apply_decision, has stored ahuman_decisionthat came from a person.
2. Suspend: stop holding anything#
The obvious first design is a loop that asks "has the underwriter answered yet?" every few seconds, or a thread that blocks on a queue. Both work in a demo. Both fail in production for the same reason: they tie the waiting run to one live process. A deploy restarts the process and the waiting run is gone. A thousand waiting runs hold a thousand threads.
The fix is to stop holding anything. To suspend a run means to save everything needed to continue, mark the run as waiting, and let the process forget it. Nothing runs while the person thinks. When the answer arrives, any process can load the saved record and continue.
A tiny example: run run-7641 of UW-006 suspends at 7318 ms of simulated time. At that moment its record says status suspended, approver role underwriter, policy version 2026.2, and one more thing: a token.
That token is the second half of the idea. An approval token is a one-time value the orchestrator issues when it suspends, and the person's decision must carry it back. It answers the question "which wait is this answer for?" Without it, the orchestrator cannot tell a decision for this wait from a decision for an earlier wait of the same run, or from an old browser tab.
Here is the code that suspends a run in the example. It is short because suspending is mostly writing things down.
def suspend(self, run: dict[str, Any], step: Step) -> None:
"""Stop holding compute; issue an approval token and wait for a person."""
if run["steps"][step.name]["status"] == "pending":
self.set_step_status(run, step.name, "start")
run["waits"] += 1
run["steps"][step.name]["attempts"] += 1
run["path"].append(step.name)
token = make_approval_token(run["run_id"], run["revision"], run["waits"])
rec = (run["steps"].get("record_recommendation", {}).get("output") or {}).get("recommendation", {})
role = core.required_role(rec.get("recommendation", "escalate"))
run["waiting"] = {"status": "suspended", "approval_token": token, "policy_version": run["policy_version"],
"approver_role": role, "since_ms": run["clock_ms"]}
self.add_span(run, step.name, "human", run["clock_ms"], None, "waiting", approver_role=role,
attempt_number=run["steps"][step.name]["attempts"])
self.set_run_status(run, "await_human")
self.event(run, "run_suspended", step.name, {"approval_token": token, "approver_role": role})
That is engine.py:Engine.suspend verbatim. Read the waiting record: it holds the token, the policy version the run used, the role that may decide, and the time the wait began. Those four fields are what the resume will check. The token is built from the run_id, the run's revision and a wait counter, so a second wait in the same run gets a different token. After this function the engine saves a checkpoint, and the process may exit.
For a backend engineer. This is the difference betweenThread.sleepand a row in a table with statusPENDING_APPROVAL, which most Spring applications already have for manual review queues. The workflow version adds two things a plain status column usually lacks: a token that ties the answer to one specific wait, and a re-check of the conditions on resume. The comparison stops holding at the event log: a status column records where the case is, not how it got there.
Diagram: One approval, start to end · the run lets go while it waits
Time runs down the page. The orchestrator stores the recommendation, saves the run with a fresh token and lets go. Nothing runs while the underwriter thinks. The decision comes back with the token; code checks it and either applies it or says why not. The agents are scripted stand-ins and time is simulated.
- Orchestrator (code) to Checkpoint store: Suspend: save run, token, policy version, role. Engine.suspend writes the waiting record and a checkpoint. The process may now exit.
- Orchestrator (code) to Underwriter cockpit: Recommendation is ready for a decision. The stored recommendation appears in the underwriter's queue with the token attached.
- Underwriter (a person): Hours or days pass; nothing runs. No thread, no connection, no memory is held for this run during the wait.
- Underwriter cockpit to Orchestrator (code): Signal: token, role, decision. The underwriter's decision travels back with the token issued for this wait.
- Orchestrator (code) to Checkpoint store: Load run; accept_resume checks five things. Used token, run waiting, token match, role, policy version. Code decides; no model takes part.
- Orchestrator (code) to Underwriter cockpit: Applied, ignored as a repeat, rejected, or sent back. Every outcome gets an answer with a reason, so the person knows whether the decision counted.
3. Resume: four answers to one click#
The underwriter clicks Approve. The underwriter cockpit sends a signal to the orchestrator: the token, the role of the person, and the decision. Now the orchestrator must decide what that signal does. This decision belongs to code, never to a model, and it is a pure function in the example: core.py:accept_resume.
def accept_resume(waiting: dict[str, Any], signal: dict[str, Any], current_policy_version: str) -> dict[str, Any]:
"""Decide what a decision signal does to a waiting run.
`waiting`: status, approval_token, consumed_tokens, policy_version, approver_role.
`signal`: token, decided_by (a role), decision.
Actions: apply, recheck_eligibility (stale approval), ignore_duplicate,
reject_not_waiting, reject_token, reject_role.
"""
if signal.get("token") in waiting.get("consumed_tokens", []):
return {"action": "ignore_duplicate", "accepted": False,
"reason": "This token was already used; the signal is acknowledged and ignored."}
if waiting.get("status") != "suspended":
return {"action": "reject_not_waiting", "accepted": False, "reason": "The run is not waiting for a decision."}
if signal.get("token") != waiting.get("approval_token"):
return {"action": "reject_token", "accepted": False,
"reason": "The token does not match the one issued for this wait."}
if ROLE_RANK.get(signal.get("decided_by"), 0) < ROLE_RANK[waiting["approver_role"]]:
return {"action": "reject_role", "accepted": False,
"reason": f"This decision needs a {waiting['approver_role']}."}
if waiting.get("policy_version") != current_policy_version:
return {"action": "recheck_eligibility", "accepted": False,
"reason": f"The run used policy {waiting.get('policy_version')}; {current_policy_version} is now in force. "
"The rules are checked again before any decision is applied."}
return {"action": "apply", "accepted": True, "reason": "Right token, authorized role, same policy version."}
The order of the checks matters, and each one exists because of a real accident. A used token comes first, so a double click is recognized even after the run has moved on. Then "is the run waiting at all?". Then the token, then the role, then the policy version. Only a signal that passes all five is applied.
What a successful call proves: given this waiting record and this signal, the decision is the one in the return value. What it does not prove: that the cockpit sent the right role. The role arrives with the signal, so in a real system it must come from the cockpit's authentication, never from a field the browser fills in.
The following script calls the function with one waiting record and four signals. It ran in a scratch copy of the example.
import core
waiting = {"status": "suspended", "approval_token": "apr-6b88a561", "consumed_tokens": [],
"policy_version": "2026.2", "approver_role": "underwriter"}
signals = [
("right token, same policy", {"token": "apr-6b88a561", "decided_by": "underwriter", "decision": "approve"}, "2026.2"),
("old token from a reload", {"token": "apr-00000000", "decided_by": "underwriter", "decision": "approve"}, "2026.2"),
("policy changed meanwhile", {"token": "apr-6b88a561", "decided_by": "underwriter", "decision": "approve"}, "2026.3"),
]
for label, signal, policy in signals:
print(f"{label:26} -> {core.accept_resume(waiting, signal, policy)['action']}")
waiting["consumed_tokens"] = ["apr-6b88a561"]
print(f"{'same click, second time':26} -> {core.accept_resume(waiting, signals[0][1], '2026.2')['action']}")
right token, same policy -> apply
old token from a reload -> reject_token
policy changed meanwhile -> recheck_eligibility
same click, second time -> ignore_duplicate
Four signals that look the same from the person's side get four different treatments. That is the whole job of the resume.
| Signal | What the orchestrator does | Why |
|---|---|---|
| Right token, authorized role, same policy | apply: resume, run apply_decision | Every condition holds |
| Token already used | ignore_duplicate: acknowledge, change nothing | A double click or a retried request (UW-905) |
| Token does not match | reject_token | An old tab, or an answer meant for an earlier wait |
| Role too low | reject_role | An escalate needs a senior_underwriter (UW-004) |
| Policy version changed | recheck_eligibility: back to the rules | A stale approval (UW-902) |
4. Exactly once: the double click#
Case UW-905 is this site's own. The underwriter clicks Approve, the network is slow, the button looks dead, and the underwriter clicks again. Two signals with the same token reach the orchestrator.
The first one is applied: the token goes into consumed_tokens, the run resumes, step 8 stores the human_decision, and the run completes. The second one finds its token in consumed_tokens and is acknowledged and ignored. The real run printed this:
$ $PY run.py --case UW-905
Quote workflow · scripted stand-in agents (no model called) · simulated time · seed 7
UW-905 · The underwriter's decision arrives twice · dag mode · run-2910 · trace 9f2d6a07c246...
path: intake > gather_evidence > check_eligibility > draft_recommendation > verify_process > record_recommendation > await_underwriter > apply_decision
fan-in: roof_assessment ok [ok], guidance ok [ok], similar_decisions ok [ok]
guard: pass (Eligibility ran and every cited rule was retrieved.)
recommendation: approve · cites - · missing - · policy 2026.2 · stored row status stored
signal: underwriter approve -> apply
signal: underwriter approve -> ignore_duplicate
human_decision: {"decided_by": "underwriter", "decision": "approve", "override": false, "reason": "Clean submission; agrees with the recommendation."}
status: completed · model calls 3 · tool calls 4 · simulated time 0.8 h (includes the wait for a person)
expected: as stated in the case
wrote 1 run file(s) in out/runs/ and out/summary.json
Note the word "acknowledged". The second request gets a polite answer, not an error. The cockpit asked "did my decision count?" and the true answer is yes. An error would make the person click a third time, or call support.
Common confusion. "Exactly once" here does not mean the signal is delivered exactly once. Networks deliver a message zero, one or several times. It means the effect is applied exactly once, because the receiver recognizes repeats. Delivery guarantees in depth belong to Agent Reliability Engineering.
5. The stale approval#
Back to Tuesday and Wednesday. In UW-902 the run suspended under 2026.2. The policy fixture moved to 2026.3 while it waited. The underwriter's signal carries the right token and the right role, so the first four checks pass. The fifth fails: the policy version recorded in the waiting record is not the version in force.
A stale approval is a decision that was correct for the world the run saw, and arrives after that world changed. The orchestrator does not apply it and does not discard the run. It sends the run back to check_eligibility, under the new version, with a new revision number. The later steps run again: a new draft, the guard, a new stored recommendation (the old row is marked superseded), and a new wait with a new token. The real output:
$ $PY run.py --case UW-902
Quote workflow · scripted stand-in agents (no model called) · simulated time · seed 7
UW-902 · The policy fixture changes while the run waits · dag mode · run-019a · trace 860dc349a2c1...
path: intake > gather_evidence > check_eligibility > draft_recommendation > verify_process > record_recommendation > await_underwriter > check_eligibility > draft_recommendation > verify_process > record_recommendation > await_underwriter
fan-in: roof_assessment ok [ok], guidance ok [ok], similar_decisions ok [ok]
guard: pass (Eligibility ran and every cited rule was retrieved.)
recommendation: decline · cites R-ROOF-01 · missing - · policy 2026.3 · stored row status stored
signal: underwriter approve -> recheck_eligibility
human_decision: none yet (waiting for: underwriter)
status: suspended · model calls 4 · tool calls 5 · simulated time 1.5 h (includes the wait for a person)
expected: as stated in the case
wrote 1 run file(s) in out/runs/ and out/summary.json
Read the path from the right. The run is suspended again, waiting for an underwriter to decide on a new recommendation, decline under 2026.3. The evidence was not gathered again: the roof age did not change, only the rules did. That choice is a design decision in this example, and it is only correct because the roof facts carry no expiry. A real workflow must decide per piece of evidence how old it may be on resume.
Who decides here? Code decides that the approval is stale, by comparing two version strings. Code decides where the run goes back to. The underwriter decides again, on the new recommendation. No model takes part in any of these decisions.
Production pitfall. The person who clicked Approve on Wednesday sees a new recommendation in the cockpit, not their own approval. The cockpit must say why: "the policy changed while this case waited; the rules were checked again". A silent change looks like a lost click.
Diagram: A stale approval · the decision arrives after the rules changed
Read left to right. The run suspended under policy 2026.2. While it waited, 2026.3 came into force. The underwriter's approval passes the token and role checks but fails the version check, so code sends the run back to the rules and a new wait begins. Versions and numbers are illustrative.
- 1 · Before the wait
- Recommendation: The underwriting agent drafted approve, the guard passed, the row was stored.
- Suspend: The waiting record keeps the policy version the run used, so a resume can compare it.
- 2 · During the wait
- Policy changes: A new version of the policy fixture lowers the roof limit. Nobody tells the waiting run.
- 3 · The signal
- Approve click: A real, authorized decision about a recommendation that was right on Tuesday.
- accept_resume: The fifth check fails: 2026.2 is not 2026.3. The action is recheck_eligibility.
- 4 · Back to the rules
- check_eligibility: The run goes back to the rules under the new version. Evidence is reused; later steps run again.
- New wait: The underwriter decides again, on the new recommendation, with a new token.
6. Nobody answers: escalation on timeout#
The third failure is silence. The underwriter is on leave. The run waits forever and the broker hears nothing.
An escalation moves a decision to a person with more authority, or to a different queue, because the first person cannot or did not decide. The quote workflow escalates for two different reasons, and it helps to keep them apart. A content escalation comes from the case: R-ROOF-03 in UW-004 says a senior underwriter must settle a disputed roof age, so required_role returns senior_underwriter from the start. A timeout escalation comes from the clock: nobody answered within the allowed time.
In the example the allowed time is 24 hours of simulated time (APPROVAL_TIMEOUT_MS in engine.py; an illustrative value). Engine.check_approval_timeout is called by a scheduler, not by the run itself, because a suspended run has no thread to notice the time. When the wait is too long, it changes the approver role to senior_underwriter, issues a new token and stays suspended. The old token stops working, so a late answer from the first underwriter is rejected with a reason instead of racing the senior underwriter's answer. The unit test test_approval_timeout_escalates_with_a_new_token checks both sides: no escalation one minute in, escalation after 25 hours.
The same idea in a coding agent. A coding agent opens a pull request and the run waits for a human reviewer. The review is the approval; the head commit is the token. If someone pushes a commit that changes the diff, an approval of the old diff is stale; GitHub's protected branches can be set to dismiss such approvals. Hooks, review and CI are covered in depth on Diff to Deploy.
7. The human decision, and the override#
When a signal is applied, step 8, apply_decision, writes the human decision: a small record of who decided (a role, never a name), what they decided, whether it went against the recommendation, and why. In UW-009 the recommendation was approve and the underwriter declined:
{"decided_by": "underwriter", "decision": "decline", "override": true, "reason": "The broker's loss run shows an open water damage claim that is not on the application."}
The override flag is computed by code (workflow.py:is_override): it is true only when the decision goes against an approve or decline recommendation. Deciding on an escalate or request_info recommendation is not an override, because those recommendations asked a person to settle the question. The reason is mandatory for an override. It is the most valuable sentence in the whole run: it is information the agent did not have, written by the person who had it.
What this is not. This page decides when a run waits and what an answer does to it. How the recommendation is shown to the underwriter, and how a checker around the model validates what it produced, belong elsewhere: checks around a model are the subject of Guardrails Engineering, and how a saved wait is stored durably is the subject of Memory Engineering.
8. Try it: play the underwriter#
The lab below suspends one run and lets you play the underwriter. Choose approve, decline, override or no answer, and switch on the three accidents: the policy version changes while the run waits, the decision arrives twice, the orchestrator process restarts during the wait. You see whether the resume is accepted, which check decided it, and what the run does next.
The same information without JavaScript, from the example's runs:
| Your action and accident | Case in the example | What the orchestrator does |
|---|---|---|
| Approve, nothing goes wrong | UW-002 | apply: human_decision stored, run completed |
Decline against an approve recommendation | UW-009 | apply: override true, the reason is stored |
| Approve, then the same click again | UW-905 | first apply, second ignore_duplicate |
Approve after the policy moved to 2026.3 | UW-902 | recheck_eligibility: back to the rules, new recommendation, new wait |
| No answer for 24 simulated hours | unit test | escalated to senior_underwriter with a new token; still suspended |
| Process restarts during the wait | UW-901 (restart mechanics) | nothing is lost: the waiting record is read back from the checkpoint |
Who decides here#
| Decision | Who |
|---|---|
| The run must wait for a person | Code: step 7 is fixed in the workflow, R-HUMAN-01 |
| Which role may decide | Code: required_role, from the recommendation |
| Whether a signal is applied, ignored, rejected, or sends the run back | Code: accept_resume |
| When silence becomes an escalation | Code: a scheduler calls check_approval_timeout |
| The decision itself | A person, the underwriter |
| What a model decides at this step | Nothing |
Checklist#
- The wait holds no thread, no connection and no process; the run is a saved record while it waits.
- Every wait issues a fresh approval token, and the decision must carry it back.
- A used token is acknowledged and ignored, never applied twice and never an error.
- The role of the decider comes from authentication, and is checked against the role the recommendation requires.
- The resume re-checks what may have changed during the wait (here
policy_version) before applying anything. - A timeout is checked by a scheduler, escalates to a named role, and invalidates the old token.
- An override stores a reason, and the person who decided is a role in the record, never a name in a log line.
- A test proves that no
human_decisionexists without a person's signal.
Races around a resumeTwo signals at once, a timeout and an answer at once
The checks in accept_resume are only correct if the read of the waiting record and the write that consumes the token happen as one step. In the example a single process handles one signal at a time, so the race cannot happen. In production two cockpit requests can reach two orchestrator workers in the same millisecond. Both read consumed_tokens empty, both apply. The usual fix is a conditional write: consume the token with a compare-and-set on the run's version number (optimistic locking) and let the loser re-read and find its token used.
The same race exists between a late answer and the timeout escalation. If the scheduler escalates while the underwriter's signal is in flight, exactly one of them must win, and the loser must see a clear reason. Issuing a new token on escalation is what makes this decidable: after escalation the first underwriter's token no longer matches, so the signal is rejected with reject_token instead of quietly overwriting the senior underwriter's decision.
A third race is the stale check itself. The policy version can change between the check and the write of the decision. A strict design records the policy version inside the same conditional write; a looser one accepts a window of milliseconds and documents it.
What to re-check on resumeVersion strings are easy; facts with a shelf life are not
The example re-checks one thing: the policy version. A real workflow has a longer list, and each item needs an explicit rule. Rules and prompts have versions, so a comparison is cheap and exact. Evidence has an age: a roof assessment from last year may still be fine, a quote valid for 30 days is not. The submission itself can change: the broker sends an amended email while the run waits. Each of these needs a decision written down in the workflow, not left to the person: go back to which step, and redo what.
A useful way to write it is a table per wait step: input, how it can change, how the change is detected, where the run goes back to. Reviewing that table is cheaper than discovering a stale approval in an audit. The site's durable workflows page explains how going back works when some later steps have side effects that must be undone.
What to look for in a frameworkA durable wait, a signal, a way to re-check
Workflow engines and agent frameworks offer pieces of this under different names: an interrupt or a signal, a wait that survives a restart, a resume with a value. Read three things in their documentation before relying on one. Does the wait survive a process restart without your own storage? How is a resume tied to one specific wait (the role of the token here)? What happens to a second resume for the same wait? The answers for each product, read on its own documentation, are on Frameworks. No framework decides for you which facts to re-check on resume; that stays your workflow's job.
In one breath#
A human approval is a wait that costs nothing: the run saves itself with a one-time token and lets go of every resource. When the decision arrives, code checks the token, the role and whether the world changed, then applies it, ignores it as a repeat, rejects it, or sends the run back to the rules. Silence is escalated by a scheduler with a new token. The model recommends; a person decides, and that decision is applied exactly once.
What comes next#
You can now pause a run for a person and resume it safely. One question was quietly assumed throughout: that the saved record survives. A suspended run is only as good as the checkpoint it was written to, and a run that is not suspended can also die in the middle of a step. The next page, durable workflows, is about checkpoints, retries, compensation and cancellation: how a run survives the process that runs it.