> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gdilabs.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Scenario execution

> This document walks a single job through the scenario runtime end-to-end: from the worker receiving a payload to the terminal ScenarioResult and the event stream that lands on the SSE channel. It's the companion to the API reference in S...

## 1. Worker dispatch

The legacy `WorkerOrchestrator` is unchanged: it still routes free vs. paid
through `routing.py` and either short-circuits to `free_model_node` or
delegates to a single `HierarchyOrchestrator` instance. The change is
that the field formerly named `hierarchy` is now a `ScenarioOrchestrator`
with the same public `run(payload, routing_decision, event_bus) ->
ScenarioResult` signature.

## 2. Orchestrator construction (per-process)

When the worker boots:

1. `ScenarioOrchestrator.__post_init__` runs once.
2. `register_with_existing_verifier(default_registry, build_verifier)`
   wires the worker's pre-constructed `BuildVerifier` under the name
   `build`.
3. `register_llm_judge(default_registry, providers)` injects the worker's
   `LlmProviders` instance into a fresh `LLMJudgeVerifier` factory.
4. `ScenarioLoader(scenarios_root).load_all()` reads every
   `knowledge-hub/scenarios/*.yaml`, validates each against the Pydantic
   schemas, and compiles every `edge.when` clause through `expr.py`. Any
   error fails fast with the source filename in the message.
5. Per-tenant overrides (when the worker has been configured with a
   `tenant_id` + `TenantOverrideProvider`) deep-merge over the base
   scenarios.
6. `PromptLevelRunner`, `VerifierLevelRunner`, `FanOutLevelRunner`,
   `HumanApprovalLevelRunner` are constructed and registered with
   `ScenarioExecutor` under their respective `kind` keys.
7. `ScenarioCompiler(scenario, dispatcher=executor).compile()` runs once for
   **every** loaded scenario; the compiled graphs land in
   `_compiled_by_id` keyed by scenario id. `run()` then picks one per
   job — see [§ 3a](#3a-per-job-scenario-selection) below. Topology rules
   (applied per scenario):
   * one node per level;
   * edges with `when: None` become unconditional `add_edge`s;
   * edges with a `when` clause are grouped per source and emitted as
     `add_conditional_edges` with a router function that evaluates the
     compiled DSL expressions in priority order and emits a
     `gate_decision` event when a branch fires;
   * levels with no outgoing edges implicitly route to `END`.

## 3a. Per-job scenario selection

The worker compiles every scenario at startup, then picks one **per job**.
`ScenarioOrchestrator._resolve_scenario_for_job(payload)` walks three
sources, highest wins:

1. **Explicit pin** — `payload.metadata["scenario_id"]`. Used by replay
   fixtures, admin tools, and the future universal-input classifier that
   wants to override the route mapping.
2. **Intent-route mapping** — `payload.metadata["intent_route"]` looked up
   in the `intent_route → scenario_id` index the loader built from each
   scenario's `intent_routes` declaration. Lookup is case-insensitive.
   Today's bundled mapping:
   `project → code_build`, `document`/`writing → document_writing`,
   `task`/`qa → quick_qa`, `research → research_brief`,
   `security_review → security_review`. Two scenarios claiming the same
   route is a hard load error — the dispatch must be unambiguous.
3. **Default** — `CEREBRUM_DEFAULT_SCENARIO` env or `code_build`. Also
   the fallback when an explicit id is unknown or a route is unmapped (the
   run continues with a warning log; a misconfigured payload must never
   break the job).

The selection appears in the `scenario_started` event as
`selection_reason`: `"explicit"`, `"intent_route:<route>"`, or
`"default"` — useful for telemetry and "why did this scenario run?"
debugging.

## 3. Job invocation

`run(payload, routing_decision, event_bus)` builds the initial
`ScenarioState`:

```python theme={null}
{
  "scenario_id": ..., "scenario_version": ..., "payload": ...,
  "routing_decision": ..., "event_bus": ...,
  "context": compose_context(payload),       # the prompt-prepended block
  "provider_candidates": ..., "chosen_provider": ..., "chosen_model": ...,
  "levels": {}, "scratchpad": Scratchpad(job_id),
  "tracker": EscalationTracker(max_attempts_per_conflict=3),
  "escalate_now": False, "verdict": "",
  "budget": scenario.budget, "budget_exceeded": False, "aborted_reason": None,
  ... # aggregate cost/token counters
}
```

Then emits `scenario_started` and invokes `compiled.invoke(initial)`.
LangGraph drives every node from the entry point (first level by YAML
order) until the graph reaches `END`.

## 4. Level dispatch

`ScenarioExecutor.run_level(state, level)` is the per-node body. It:

1. Skip-fast when `state.aborted_reason` is set (budget kill switch
   short-circuit). Returns a synthetic `skipped` level state and emits
   `level_finished` with `ok=true, duration_ms=0`.
2. Emits legacy `node_started` and new `level_started` (with
   `level_id`, `role`, `kind`, `attempt`, `model_constraint_allow`).
3. Calls the kind-specific runner.
4. Emits `node_finished` + `level_finished` with cumulative duration,
   cost, and the output preview (first 200 chars from the level's
   delta).

### `kind: prompt` (`PromptLevelRunner`)

1. Resolves the prompt builder via `get_prompt_builder(level.prompt_ref)`
   and builds a `BuiltPrompt`.
2. Emits `agent_handoff` + `prompt_sent` when the builder declared them
   (suppressed on retries by the builder itself).
3. Resolves the model via `resolve_model(level.model_constraint, routing_default)`
   and emits `routing_decision` with the resolved tier / provider /
   model / `constraint_source`.
4. Wraps the call in `event_bus.heartbeat(...)` for L4-class long
   generations.
5. Calls `_call_with_ask_user(...)` — same `ask_user` clarification loop
   as the legacy hierarchy.
6. Records cost + tokens into the running aggregates, emits
   `budget_consumed`.
7. Runs `state.budget` check; on exceeded, sets `aborted_reason` and
   emits `budget_exceeded`.
8. Per-level post-processing:
   * `l1_brief`: emits a Markdown brief (goal + acceptance criteria)
     directly — no JSON envelope, no collapse step; its prose *is* the
     level output L2/L3 read.
   * `l3_accept` / `fact_check`: extract the verdict via
     `_verdict_from_l3_output` — a trailing ` ```control ` fence first
     (`worker/runtime/control_block.py`), then a lenient JSON path
     (plain `{verdict}` or `extra.verdict`). When neither yields a clean
     PASS/FAIL, a single bounded **repair-retry** re-asks with the
     contract spelled out before falling through. The verdict is written
     to `state.verdict`; FAIL registers a failure with the escalation
     tracker and sets `escalate_now` when the conflict budget is hit.
   * `polish` / `synthesis` / `answer` / `final_report`: terminal-output
     levels; their `result.output` becomes `state.final_output`.
9. If the level itself declared verifiers (e.g., an `llm_judge` rubric on
   `l3_accept` or `fact_check`), the runner invokes them through the
   `VerifierRunner` and folds the per-verifier outcomes into the level
   state alongside the prompt result.

### `kind: tool_loop` (`ToolLoopLevelRunner`)

`l4_execute` and `l2_escalate` are **agentic file-tool loops** — the model
mutates the project workspace through tool calls executed by the worker
(Claude Code style: the **filesystem is the artifact**, not the message
transcript). Downstream levels receive the *touched paths*, never file
bodies.

1. Builds the task prompt via `prompt_ref` (no Markdown file-fence
   contract — the runtime provides the tools) and emits the
   `agent_handoff` / `routing_decision` events like `prompt`.
2. Loops up to `Level.metadata["max_iterations"]` (default 40):
   * **paid** transport — `LlmProviders.call_model_turn(messages,
     tools=anthropic_tools())` returns text plus native `tool_use` blocks;
   * **free** transport — only when the *first* paid turn fails and the
     level allows `fallback_outside_allow`: the textual ` ```tool ` /
     ` ```tool_result ` protocol over `call_free_model`.
3. Each tool call (`read_file` / `write_file` / `edit_file` / `list_dir`
   / `delete_file` / `finish`) is dispatched via
   `worker/runtime/file_tools.execute_tool` against `_project_cwd`,
   sandboxed by `code_applier._safe_relative`. Every call streams a
   `tool_call` + `tool_result` event; writes also emit `file_created`.
4. The loop ends on `finish`, an empty turn, `max_iterations`, the
   scenario budget, or a user **Stop** (`bus.is_cancelled()`, checked per
   turn). Every turn's usage folds into the running aggregates.
5. Writes the level state: `output` = the `finish` summary,
   `touched_paths` = the runner's authoritative write-tool set. Terminal
   tool-loop levels (`l2_escalate`) also set `state.final_output`.

### `kind: verifier` (`VerifierLevelRunner`)

1. The `l4_execute` tool loop already wrote its changes to disk, so there
   is **no file-apply step** — the runner builds a `VerifierContext` with
   the project workspace, payload, and scratchpad and runs the declared
   verifiers directly.
2. Emit `verifier_started` per declared verifier.
3. Invoke `VerifierRunner.run(level.verifiers, ctx)`. Mode is
   `serial`, `parallel`, or `race` (see
   [scenarios-runtime.md][component]).
4. Emit `verifier_finished` per result.
5. Stash a one-line `build_result_summary` on the scratchpad (pass or
   fail) so the `l3_accept` prompt can review plan-vs-done without
   re-reading the workspace.
6. On failure, format **every** failed verifier into a feedback string —
   the build stderr tail plus any other gate's reasons (e.g. the
   `route_reachability` orphan/misplaced-route list) — and stash it under
   `build_feedback` so the next L4 retry fixes all blockers. Register
   failure with the escalation tracker on conflict id `build_failure`.
7. Emit `build_verify_passed` or `build_verify_failed` for legacy
   compatibility.

`code_build`'s `l3_build_verify` runs two verifiers: `build` (install +
`next build`) and **`route_reachability`** — a deterministic, no-LLM gate
that every Next.js App Router route is navigable from `/`. It flags
*orphan* routes (a `page.*` nothing reachable from the home page links to)
and *misplaced* pages (a `page.*` written into a parallel `app/` tree such
as `apps/frontend/app/...`, which Next never routes), following
`Link`/`href`/`router.push`/`redirect` targets transitively through
imported components. Dynamic (`[id]`) and `api` routes are exempt, and a
project with no `app/` dir skip-passes (so non-Next builds and the replay
harness are untouched). The `l3_build_verify → l3_accept` edge ANDs both
verifiers, so a navigability failure routes back to L4 with the offending
routes — same retry/escalation budget as a build failure. `code_build_strict`
adds it alongside `typecheck` + `lint`.

### `kind: aggregator` (`FanOutLevelRunner`)

1. Dispatches each child in `level.fan_out` through the same
   `ScenarioExecutor` via a `ThreadPoolExecutor`. Each child runs through
   its own kind runner (so a child can itself be `prompt`, `verifier`,
   or `human_approval`).
2. Merges child level states into the parent's `levels` slot so the
   synthesis prompts can read them by id and the audit log retains every
   child output.
3. Hands the `ChildOutcome[]` to the aggregator named in
   `level.fan_in.aggregator` (`majority_pass`, `weighted_score`,
   `all_pass` by default).
4. Sets the parent level's `output` to the aggregator's summary and
   sets `state.verdict` to the aggregator's `PASS`/`FAIL` so the gate
   router can branch off it.
5. Checks the budget against the sum of children's cost (parallel
   children each see only their own slice before the merge, so the
   parent's post-merge check is what stops a fan\_out from slipping past
   the cap).

### `kind: human_approval` (`HumanApprovalLevelRunner`)

1. Builds the human-visible question via the level's `prompt_ref` builder.
2. Emits `ask_user` with `call_id`, `interrupt_id`, `approver_role`, and
   the level's `output_schema` (or a default `{approved, comment}`
   schema).
3. Calls `langgraph.types.interrupt(...)` which raises a `GraphInterrupt`
   the worker's checkpointer catches; the run pauses until
   `Command(resume=...)` lands.
4. On resume, packages `{approved, comment}` into the level's
   `output_parsed` and writes `state.verdict = PASS|FAIL`.
5. Emits `tool_result` with the approval so the timeline shows the
   decision landed.
6. On reject, registers failure with the tracker under conflict
   `human_approval:<level_id>` so a chain of rejections eventually trips
   `escalate_now`.

When the langgraph runnable context is missing (direct unit-test
invocation outside `graph.invoke()`), the runner auto-approves with a
loud log so the runtime stays testable without a real checkpointer.

## 5. Gate routing

After each level returns, LangGraph dispatches to the source's outgoing
edges. For conditional edges:

* The compiler's router evaluates each compiled expression in
  `priority` order against a `MappingContext` built from `state`. The
  context surfaces `verifier.<name>.<field>` from the current level's
  verifier results, `verdict`, `escalate_now`, `confidence`,
  `level.<id>.<field>`, and `scratchpad.<key>`.
* The first matching edge wins. The router emits `gate_decision` with
  `source_level`, `target_level`, `matched_expression`, and `reason`.
* When nothing matches, the router falls through to `END` with a loud
  log entry — scenarios should declare a default `when: None` edge to
  avoid this.
* **Kill-switch short-circuit:** when `state.aborted_reason` is set
  (budget cap or user Stop), the router routes straight to `END` —
  otherwise the stale state of skipped levels would route a retry edge
  back into a loop (e.g. `l3_build_verify → l4_execute` forever).

## 6. Termination

When the graph reaches `END`, the orchestrator:

1. Reads the terminal state and synthesizes a `ScenarioResult` (output =
   `state.final_output` or fallback to `levels[l4_execute].output`).
   It also carries `workspace_owned` (true when the scenario has a
   `tool_loop` level), the `l3_build_verify` outcome, and `aborted_reason`
   out to the thread runner.
2. Emits `scenario_finished` with `status: completed | aborted |
   cancelled`, `aborted_reason`, `total_cost_usd`, `total_tokens`,
   `total_duration_ms`, and `levels_run`. `cancelled` is a user Stop;
   `aborted` is a budget cap or a crash.

The frontend's `ScenarioRunTimeline` consumes the event stream and
derives the per-level / per-verifier / gate / budget views via the
helpers in `apps/frontend/lib/workflow/scenario-events.ts`. The user
sees the full story of the run — which model ran which level, which
verifier passed or failed with what score, which branch the gate took —
without polling any additional endpoint.

## 7. User Stop

A user can halt a running job from the workflow composer's Stop button
at any step. The mechanism is a Redis flag, polled at safe points — no
new thread, no signal handler:

1. `POST /v1/jobs/:id/cancel` (mother-ai) sets `cerebrum:job:{id}:cancel`
   (1 h TTL), marks the job terminal `cancelled` in the Redis status hash,
   and best-effort marks `prompt_threads.status='cancelled'` in Postgres —
   so the UI unsticks immediately even if no worker is holding the job.
2. The worker observes the flag via `EventBus.is_cancelled()` (a memoized
   `EXISTS` check): the executor polls it at **every level boundary**,
   `ToolLoopLevelRunner` once **per tool turn**, and the Anthropic
   streaming loop cuts a doomed generation short.
3. On detection the executor sets `aborted_reason="cancelled_by_user"` —
   the same kill-switch path as a budget abort: all remaining levels
   skip, the gate router short-circuits to `END`, and `scenario_finished`
   fires with `status: "cancelled"`.
4. A cancelled run **discards partial work**: `main.py` skips
   `commit_and_push` and marks the job + prompt `cancelled`. Only a new
   prompt in the same workflow starts a fresh run from L1.

(Stop only halts an *actively-running* worker; orphaned jobs — where the
worker died without recovery — are a separate, tracked infra gap.)

## 8. Replay path

For CI / regression detection, the same pipeline runs against
`CassetteProviders` instead of live `LlmProviders`. The cassette is a
list of recorded `ModelCallResult` dicts on a golden fixture; every
prompt-runner / `call_model_turn` / llm\_judge call pops the next entry in
order — a `tool_loop` cassette entry carries the `tool_calls` the loop
dispatches. The harness provisions an ephemeral git workspace for any
scenario with a `tool_loop` level. Verifier results are byte-for-byte
deterministic (subprocess output or schema validation), so the harness
diffs the run's verifier scores + verdict + cost against the fixture's
`expected` block. The CI gate fails on any regression beyond declared
thresholds.

[component]: ../components/scenarios-runtime.md