Scenario execution

​

1. Worker dispatch

​

2. Orchestrator construction (per-process)

1. ScenarioOrchestrator.__post_init__ runs once.
2. register_with_existing_verifier(default_registry, build_verifier) wires the worker’s pre-constructed BuildVerifier under the name build.
3. register_llm_judge(default_registry, providers) injects the worker’s LlmProviders instance into a fresh LLMJudgeVerifier factory.
4. ScenarioLoader(scenarios_root).load_all() reads every knowledge-hub/scenarios/*.yaml, validates each against the Pydantic schemas, and compiles every edge.when clause through expr.py. Any error fails fast with the source filename in the message.
5. Per-tenant overrides (when the worker has been configured with a tenant_id + TenantOverrideProvider) deep-merge over the base scenarios.
6. PromptLevelRunner, VerifierLevelRunner, FanOutLevelRunner, HumanApprovalLevelRunner are constructed and registered with ScenarioExecutor under their respective kind keys.
7. ScenarioCompiler(scenario, dispatcher=executor).compile() runs once for every loaded scenario; the compiled graphs land in _compiled_by_id keyed by scenario id. run() then picks one per job — see § 3a below. Topology rules (applied per scenario):
   • one node per level;
   • edges with when: None become unconditional add_edges;
   • edges with a when clause are grouped per source and emitted as add_conditional_edges with a router function that evaluates the compiled DSL expressions in priority order and emits a gate_decision event when a branch fires;
   • levels with no outgoing edges implicitly route to END.

​

3a. Per-job scenario selection

1. Explicit pin — payload.metadata["scenario_id"]. Used by replay fixtures, admin tools, and the future universal-input classifier that wants to override the route mapping.
2. Intent-route mapping — payload.metadata["intent_route"] looked up in the intent_route → scenario_id index the loader built from each scenario’s intent_routes declaration. Lookup is case-insensitive. Today’s bundled mapping: project → code_build, document/writing → document_writing, task/qa → quick_qa, research → research_brief, security_review → security_review. Two scenarios claiming the same route is a hard load error — the dispatch must be unambiguous.
3. Default — CEREBRUM_DEFAULT_SCENARIO env or code_build. Also the fallback when an explicit id is unknown or a route is unmapped (the run continues with a warning log; a misconfigured payload must never break the job).

​

3. Job invocation

{
  "scenario_id": ..., "scenario_version": ..., "payload": ...,
  "routing_decision": ..., "event_bus": ...,
  "context": compose_context(payload),       # the prompt-prepended block
  "provider_candidates": ..., "chosen_provider": ..., "chosen_model": ...,
  "levels": {}, "scratchpad": Scratchpad(job_id),
  "tracker": EscalationTracker(max_attempts_per_conflict=3),
  "escalate_now": False, "verdict": "",
  "budget": scenario.budget, "budget_exceeded": False, "aborted_reason": None,
  ... # aggregate cost/token counters
}

​

4. Level dispatch

1. Skip-fast when state.aborted_reason is set (budget kill switch short-circuit). Returns a synthetic skipped level state and emits level_finished with ok=true, duration_ms=0.
2. Emits legacy node_started and new level_started (with level_id, role, kind, attempt, model_constraint_allow).
3. Calls the kind-specific runner.
4. Emits node_finished + level_finished with cumulative duration, cost, and the output preview (first 200 chars from the level’s delta).

​

kind: prompt (PromptLevelRunner)

1. Resolves the prompt builder via get_prompt_builder(level.prompt_ref) and builds a BuiltPrompt.
2. Emits agent_handoff + prompt_sent when the builder declared them (suppressed on retries by the builder itself).
3. Resolves the model via resolve_model(level.model_constraint, routing_default) and emits routing_decision with the resolved tier / provider / model / constraint_source.
4. Wraps the call in event_bus.heartbeat(...) for L4-class long generations.
5. Calls _call_with_ask_user(...) — same ask_user clarification loop as the legacy hierarchy.
6. Records cost + tokens into the running aggregates, emits budget_consumed.
7. Runs state.budget check; on exceeded, sets aborted_reason and emits budget_exceeded.
8. Per-level post-processing:
   • l1_brief: emits a Markdown brief (goal + acceptance criteria) directly — no JSON envelope, no collapse step; its prose is the level output L2/L3 read.
   • l3_accept / fact_check: extract the verdict via _verdict_from_l3_output — a trailing ```control fence first (worker/runtime/control_block.py), then a lenient JSON path (plain {verdict} or extra.verdict). When neither yields a clean PASS/FAIL, a single bounded repair-retry re-asks with the contract spelled out before falling through. The verdict is written to state.verdict; FAIL registers a failure with the escalation tracker and sets escalate_now when the conflict budget is hit.
   • polish / synthesis / answer / final_report: terminal-output levels; their result.output becomes state.final_output.
9. If the level itself declared verifiers (e.g., an llm_judge rubric on l3_accept or fact_check), the runner invokes them through the VerifierRunner and folds the per-verifier outcomes into the level state alongside the prompt result.

​

kind: tool_loop (ToolLoopLevelRunner)

1. Builds the task prompt via prompt_ref (no Markdown file-fence contract — the runtime provides the tools) and emits the agent_handoff / routing_decision events like prompt.
2. Loops up to Level.metadata["max_iterations"] (default 40):
   • paid transport — LlmProviders.call_model_turn(messages, tools=anthropic_tools()) returns text plus native tool_use blocks;
   • free transport — only when the first paid turn fails and the level allows fallback_outside_allow: the textual ```tool / ```tool_result protocol over call_free_model.
3. Each tool call (read_file / write_file / edit_file / list_dir / delete_file / finish) is dispatched via worker/runtime/file_tools.execute_tool against _project_cwd, sandboxed by code_applier._safe_relative. Every call streams a tool_call + tool_result event; writes also emit file_created.
4. The loop ends on finish, an empty turn, max_iterations, the scenario budget, or a user Stop (bus.is_cancelled(), checked per turn). Every turn’s usage folds into the running aggregates.
5. Writes the level state: output = the finish summary, touched_paths = the runner’s authoritative write-tool set. Terminal tool-loop levels (l2_escalate) also set state.final_output.

​

kind: verifier (VerifierLevelRunner)

1. The l4_execute tool loop already wrote its changes to disk, so there is no file-apply step — the runner builds a VerifierContext with the project workspace, payload, and scratchpad and runs the declared verifiers directly.
2. Emit verifier_started per declared verifier.
3. Invoke VerifierRunner.run(level.verifiers, ctx). Mode is serial, parallel, or race (see scenarios-runtime.md).
4. Emit verifier_finished per result.
5. Stash a one-line build_result_summary on the scratchpad (pass or fail) so the l3_accept prompt can review plan-vs-done without re-reading the workspace.
6. On failure, format every failed verifier into a feedback string — the build stderr tail plus any other gate’s reasons (e.g. the route_reachability orphan/misplaced-route list) — and stash it under build_feedback so the next L4 retry fixes all blockers. Register failure with the escalation tracker on conflict id build_failure.
7. Emit build_verify_passed or build_verify_failed for legacy compatibility.

​

kind: aggregator (FanOutLevelRunner)

1. Dispatches each child in level.fan_out through the same ScenarioExecutor via a ThreadPoolExecutor. Each child runs through its own kind runner (so a child can itself be prompt, verifier, or human_approval).
2. Merges child level states into the parent’s levels slot so the synthesis prompts can read them by id and the audit log retains every child output.
3. Hands the ChildOutcome[] to the aggregator named in level.fan_in.aggregator (majority_pass, weighted_score, all_pass by default).
4. Sets the parent level’s output to the aggregator’s summary and sets state.verdict to the aggregator’s PASS/FAIL so the gate router can branch off it.
5. Checks the budget against the sum of children’s cost (parallel children each see only their own slice before the merge, so the parent’s post-merge check is what stops a fan_out from slipping past the cap).

​

kind: human_approval (HumanApprovalLevelRunner)

1. Builds the human-visible question via the level’s prompt_ref builder.
2. Emits ask_user with call_id, interrupt_id, approver_role, and the level’s output_schema (or a default {approved, comment} schema).
3. Calls langgraph.types.interrupt(...) which raises a GraphInterrupt the worker’s checkpointer catches; the run pauses until Command(resume=...) lands.
4. On resume, packages {approved, comment} into the level’s output_parsed and writes state.verdict = PASS|FAIL.
5. Emits tool_result with the approval so the timeline shows the decision landed.
6. On reject, registers failure with the tracker under conflict human_approval:<level_id> so a chain of rejections eventually trips escalate_now.

​

5. Gate routing

• The compiler’s router evaluates each compiled expression in priority order against a MappingContext built from state. The context surfaces verifier.<name>.<field> from the current level’s verifier results, verdict, escalate_now, confidence, level.<id>.<field>, and scratchpad.<key>.
• The first matching edge wins. The router emits gate_decision with source_level, target_level, matched_expression, and reason.
• When nothing matches, the router falls through to END with a loud log entry — scenarios should declare a default when: None edge to avoid this.
• Kill-switch short-circuit: when state.aborted_reason is set (budget cap or user Stop), the router routes straight to END — otherwise the stale state of skipped levels would route a retry edge back into a loop (e.g. l3_build_verify → l4_execute forever).

​

6. Termination

1. Reads the terminal state and synthesizes a ScenarioResult (output = state.final_output or fallback to levels[l4_execute].output). It also carries workspace_owned (true when the scenario has a tool_loop level), the l3_build_verify outcome, and aborted_reason out to the thread runner.
2. Emits scenario_finished with status: completed | aborted | cancelled, aborted_reason, total_cost_usd, total_tokens, total_duration_ms, and levels_run. cancelled is a user Stop; aborted is a budget cap or a crash.

​

7. User Stop

1. POST /v1/jobs/:id/cancel (mother-ai) sets cerebrum:job:{id}:cancel (1 h TTL), marks the job terminal cancelled in the Redis status hash, and best-effort marks prompt_threads.status='cancelled' in Postgres — so the UI unsticks immediately even if no worker is holding the job.
2. The worker observes the flag via EventBus.is_cancelled() (a memoized EXISTS check): the executor polls it at every level boundary, ToolLoopLevelRunner once per tool turn, and the Anthropic streaming loop cuts a doomed generation short.
3. On detection the executor sets aborted_reason="cancelled_by_user" — the same kill-switch path as a budget abort: all remaining levels skip, the gate router short-circuits to END, and scenario_finished fires with status: "cancelled".
4. A cancelled run discards partial work: main.py skips commit_and_push and marks the job + prompt cancelled. Only a new prompt in the same workflow starts a fresh run from L1.

​

8. Replay path