Skip to main content

Harness Automation

A harness turns the manual "Claude drafts a PRD → Codex reviews → Claude fixes → decompose into stages → run each stage with review/fix cycles → commit behind gates" loop into a first-class, governed, observable object in Jentrix. It records the PRD, plan registry, stage plans, reusable prompts, reviews, decisions, stage executions, branches, commits, pull requests, and evidence — and governs the gates and freezes — while Claude Code, Codex, GitHub, and the external runner do the work. Jentrix records and governs; it never runs a build, test, deploy, or git command itself.

The harness is built entirely on the layers below it — it adds one net-new spine (HarnessRun / HarnessStage) and reuses everything else: the agent-workflow loop engine (plan → review → execute → review), the Software Delivery OS governed push path and delivery graph, Work Orders, Projects & decisions, the policy engine, and the credential broker. It does not reimplement any of them.

This guide is the runtime reference. The product requirements are in the Harness Automation PRD.

The harness spine

A Harness Run owns one end-to-end automation. It is workspace-scoped. The external records it coordinates — the anchor Task, the accepted Project / Spec / SpecVersion, and the Runbook templates it draws from — are loosely linked by id (no foreign keys — the M5 product-layer convention). Its own substrate rows (Stages, Artifacts, Findings, and the registry snapshots) are real cascade-scoped children of the run:

Harness Run  (status: DRAFT → AUTHORING → PLANNING → READY → EXECUTING → COMPLETED)
   ├─ anchor Task          (the work item the harness drives)
   ├─ Project / Spec / SpecVersion   (the accepted PRD)
   ├─ Stages               (ordered HarnessStage, 1:1 with a Work Order each)
   ├─ Artifacts            (PRD candidates, plan registry, prompts, reviews, evidence)
   ├─ Findings             (normalized review findings, with resolution + verification)
   ├─ Runbooks             (versioned, checksummed prompt templates)
   └─ Registry snapshots   (the generated parent-plan registry)

Each Harness Stage carries an ordinal number, an acceptance-criteria set traced to spec criteria, a 1:1 Work Order (its bounded brief), an optional work-lease while executing, a branch, and a status that walks TODO → PLANNING → PLAN_REVIEW → READY → EXECUTING → EXECUTION_REVIEW → FIXING → VALIDATED → COMMIT_GATE → COMMITTED (with BLOCKED / SKIPPED off-ramps).

The registry is generated from state. There is no hand-maintained progress table: get_harness returns the live registry projection computed by the first-undone-stage resolver, not a stored markdown blob. HarnessRun.currentStageId is a denormalized cache rewritten on every transition; a unit test asserts cache == resolver(). A separately stored registry snapshot (for in-repo drift detection) is checksummed and surfaces in the HARNESS_REGISTRY_DRIFT queue when it diverges.

The lifecycle

The harness runs four loops on the same agent-workflow engine — each is a WorkflowTemplate (data, not a second engine) with a distinct terminal meaning:

  1. Authoring (authoring, terminal spec_validated). Claude synthesizes a PRD, Codex reviews, Claude fixes, until the plan-approval gate is approved. The accept-gate side effect promotes the synthesized PRD to an accepted SpecVersion and advances the run AUTHORING → PLANNING. Emits harness.spec_validated. PRD coverage is citation-enforced (AGE-521): every definitionOfDone criterion cites its PRD source inline ([PRD §<section>]) and every deliberately excluded PRD section appears in nonGoals with a reason — the reviewer blocks an uncited, unexcluded section, and the human's spec-gate read IS the coverage check (PRD → spec is the pipeline's only un-mechanized hop). Downstream, stage planners and reviewers work under the stage scope contract (AGE-520): the accepted spec is read-only global context, each stage answers ONLY for its own traced criteria, and sibling-owned criteria are follow-ups, never blockers.
  2. Decomposition (decomposition, terminal plan_validated). The validated spec becomes ordered stages + 1:1 Work Orders. The harness path start_harness_decomposition opens the decomposition loop on a PLANNING run (spec validated + pinned); the planner submits the proposed ordered stages through submit_plan's harnessStages payload (each stage tracing the accepted spec criteria it delivers), keyed to the submitting job and persisted as the exact proposal the accept gate mints (the structured payload drives the mint; the reviewer/human review the plan version that the same submit freezes — a structured proposal read surface is H10.8). Spec coverage is enforced at the accept gate: approving the plan gate validates — in executable code, before the gate is consumed AND again inside the mint transaction, after reading that job's persisted proposal back — that every accepted spec criterion is traced by ≥1 stage, then mints the ordered stages + 1:1 Work Orders and advances the run PLANNING → READY. An uncovered plan is rejected, naming the orphaned criteria, and the gate stays pending — a run can never reach READY with an incomplete plan, and the gate is never approved-then-failed. Re-decomposition archives and supersedes obsolete stages (lineage preserved). Emits harness.plan_validated. The generic request_agent_workflow path cannot start decomposition — it is always harness-bound, so the accept gate always has a run to advance and a proposal to mint. The hand-authored alternative (create_harness_stage, ADMIN only) appends to the same ordered stage list carrying the same full plan a decomposition proposal does (acceptance criteria + traced spec criteria + expected branch behavior + suggested tests + risk notes), but bypasses coverage enforcement, so it is a fallback, not the canonical path. A non-draft stage always gets executable backing: a 1:1 Work Order, auto-minted when workOrderId is omitted, so it can be passed to request_harness_stage_run (the run must be task-anchored and non-terminal; a supplied workOrderId must back the run's anchor task). It is "has executable backing", not "launchable this instant" — the normal serial ordering + run/stage status still apply at launch. Pass draftOnly: true for a deliberate non-executable placeholder — no Work Order is minted, request_harness_stage_run rejects it as a draft, and list_harness_stages / the UI registry mark it launchable: false (a draft never masquerades as launchable). A pre-H10.5 sparse stage with no Work Order is treated the same way — a non-launchable draft, not a silent launch-trap.
  3. Stage execution (stage_execution, terminal stage_ready_to_commit). Since the M19 remediation (AGE-781) a stage attempt is execute → review only — there is no per-stage plan loop. Every attempt opens directly at EXECUTING: the stage's plan is the decomposition's reviewed, frozen contract (criteria + the per-stage approach brief + tests + commands), which is exactly what the executor is prompted with, and the decomposition review is the last review before execution (planning happens ONCE, over the whole multi-stage plan, while it is still amendable — the per-stage plan loop could only re-derive the frozen contract or deadlock against it, which is what the M19 failure was). This applies on requireHumanGates runs too: the human's plan gate is the decomposition gate; per stage the human gates remain final acceptance and commit. The attempt runs under a stage lease, heartbeats, an iteration budget, and evidence requirements: a stage reaches VALIDATED only with an execution summary, a diff/patch, and test evidence, and no unresolved critical/high finding. The reviewer is fed real evidence plus its own prior-round findings (the runtime $PRIOR_ROUNDS review-continuity context) so it converges instead of oscillating. Review verdicts are severity-gated (see agent-workflows): every finding carries a required 0–100 severity, and a CHANGES_REQUESTED not backed by a blocking (≥ 60) finding — new or still open from a prior round — is recorded as APPROVED with the findings kept as follow-ups, so nit-level observations can never burn the stage's convergence budget. Reviewer turns get real evidence: get_agent_job inlines the latest diff/test bodies (bounded + redacted) as harnessReview.inlineEvidence, and the reference runner hands the Codex reviewer the stage checkout read-only, so the verdict is grounded in the tree rather than the executor's prose. Convergence is bounded three ways: a strict stage-wide six-review-round cap (counted across every attempt since the stage's last cap park — EXECUTION-review rounds AND human final-acceptance rejections both consume it) parks further rejections BLOCKED for a human to accept/waive the open findings — the sweep never auto-retries a cap park, and the human's retry re-arms the budget (a retry alone never does); a reviewer verdict of "this stage AS PLANNED cannot satisfy its criteria" (returnToPlan) parks immediately as a scope dispute — no executor round can fix scope, a human re-decomposes, waives, or skips; and immutable attempt caps (AGE-782) bound everything: at most 4 retries per stage and 3 authoring / 3 decomposition jobs per run, code constants that no policy, scope, or operator ruling can raise — past a cap, the only moves left are skip, re-decompose on a fresh run, or cancel.
  4. Commit (Stage commit boundary). A VALIDATED stage crosses the commit gate and becomes COMMITTED, which unblocks the next stage. Terminal acceptance: committing the final stage is the run's acceptance — there is no separate run-level final gate. Committing the last stage moves the run EXECUTING → COMPLETED in the SAME transaction (H10.7) and emits harness.completed; the run then drops out of the stuck/active operator queues, the dashboard counts it as completed, and the playbook miner can learn from it.

Every loop reuses the engine's existing claim / heartbeat / submit / turn-guard / convergence-guard / human-gate machinery. Exceeding a loop's iteration budget auto-blocks to a human (harness.blocked).

The runnable harness contract

This is the shipped end-to-end flow. Jentrix records and governs each step; an external runner and the agents do the actual git/build/test work.

create anchored harness            (DRAFT — anchor task + repo captured)
  → assign planner + reviewer agents
  → start authoring                (DRAFT → AUTHORING; opens the authoring loop)
  → approve the spec/plan gate      (AUTHORING → PLANNING; pins the SpecVersion)
  → start decomposition            (opens the decomposition loop on the pinned spec)
  → approve the covered plan gate   (PLANNING → READY; mints ordered stages + 1:1 Work Orders)
  → stages execute in order        (READY → EXECUTING; each stage attempt runs execute → review off the frozen stage plan)
  → commit each stage behind its gate (… → COMMITTED, which unblocks the next stage)
  → final stage commit completes the run (EXECUTING → COMPLETED; emits harness.completed)

Orchestration mode. Advancing from one stage to the next has two modes, controlled by the run's autoAdvanceStages policy (M11; Settings tab or set_harness_policy):

  • Auto (default, autoAdvanceStages: true). When a stage becomes launchable (harness.stage_ready — after decomposition mints stages and after each stage commit), the run launches it directly, deduped against an already-active stage job and acting as the run's creator. Auto-advance never approves a gate — it only starts the next stage's loop; any gate that parks still stops for a person. A failed auto-launch degrades to manual (the stage stays launchable). The workspace-wide harness.stage_ready automation rule still works and composes with this — both funnel through the same guarded launch op. Reliability: the in-request fast path is best-effort (serverless after()); the cron sweep re-derives intent from durable state every ~5 minutes and retries any auto-resolve/auto-launch the fast path dropped, so auto mode is eventually-consistent even across dropped callbacks. Decomposition auto-start (AGE-309). The same policy governs the run-level authoring→decomposition hop: once the spec gate resolves and the run sits in PLANNING with a pinned spec, the sweep opens the run's first decomposition loop itself — as the run's creator with source automation, through the same guarded start op as the manual path (still-PLANNING CAS, live-duplicate rejection, agent-role resolution). Only the first: a prior decomposition loop in any non-archived state (blocked, cancelled, or done) leaves any restart to a human, and a dropped stage-mint side effect is the AGE-295 outbox's to redeliver, never a fresh loop's. With autoAdvanceStages: false the start stays the operator's (start_harness_decomposition).
  • Manual (autoAdvanceStages: false). After a stage commits, the run surfaces a single clear "Start current stage" control (UI) / request_harness_stage_run call (MCP). The operator launches each stage explicitly. Nothing auto-advances.

Blocked-stage auto-retry (autoRetryLimit, default 2). A stage that parks BLOCKED for a mechanical reason — the iteration budget ran out, the runner crashed, a heartbeat lapsed — no longer waits for an operator on an autonomous run: while requireHumanGates is false and the stage's retryCount is below the run's autoRetryLimit, the cron sweep retries it through the same guarded retryStage op as the manual path (run-EXECUTING guard, first-undone-stage serial guard, BLOCKED CAS, the decision guard), acting as the run's creator with source automation. The fresh attempt gets a fresh iteration budget and the prior attempt's findings via review continuity — the same monotone recovery an operator would perform by hand. If heartbeat expiry already blocked the newest job but its best-effort stage-park follow-up was interrupted, that same retry operation first replays the guarded park and then retries; it never edits around a live, superseded, or taken-over attempt. Boundaries:

  • Decision blocks are never auto-retried — a stage blocked on an unresolved BLOCKING decision is a human's to accept (resumeStage), and the sweep skips it.
  • Neither is a park a retry cannot possibly clear. A stage demanding an evidence tier this deployment cannot attest, and a stage whose roles have no eligible worker, both stay parked for a human: relaunching would reach the identical refusal and spend the shared budget reproducing it. Both are re-derived from live state every sweep rather than stored as a flag, so on an unmanaged run fixing the cause IS the fix — assign or reactivate a worker and the next sweep retries with nothing to clear by hand. On a managed run the sweep does not touch the stage at all, so its parks are cleared by an operator restarting the stage.
  • The budget is shared. Every retry — manual retry_harness_stage included — increments the stage's retryCount (visible in list_harness_stages detailed output and the stage activity payload), so automation never piles attempts on top of a human already driving recovery. Once retryCount ≥ autoRetryLimit the stage parks for a human exactly as before (and stays visible in STUCK_HARNESSES).
  • The hard ceiling is immutable (AGE-782). Above autoRetryLimit sits HARD_STAGE_RETRY_LIMIT = 4 — a code constant, not a policy: retryStage refuses every caller (UI, MCP, sweep) past it, the sweep's budget check clamps to min(autoRetryLimit, 4), and no set_harness_policy field, env var, or admin ruling raises it. A stage that has burned 5 attempts is not going to converge by piling on a sixth — the remaining moves are skipping the stage, re-decomposing the work on a fresh run, or cancelling. The same rule bounds the run-level loops: at most 3 authoring jobs and 3 decomposition jobs per run, ever.
  • autoRetryLimit: 0 disables the behavior; requireHumanGates: true also suspends it (a human-gated run wants humans deciding on blocks too).
  • With autoAdvanceStages: false, an auto-retry still only re-arms the stage to READY — the launch stays manual, per that policy.

Locked product decisions (H10). These remain the operative product and security boundaries:

  • Terminal acceptance is the final stage commit. There is no separate run-level final gate; the run reaches COMPLETED when its last live stage commits.
  • get_harness output stays frozen. It returns the live registry projection only. Pending gates, active/stuck runs, and launchability blockers are read through companion callslist_harness_stages (per-stage gate/branch/commit/evidence), get_agent_job / list_agent_jobs (loop phase + approval gates), and list_control_tower_items (the harness queues). Tool and docs text must point MCP users at those companions instead of widening get_harness. Agent-role (planner/reviewer) readiness has no read tool — it is not exposed over MCP; a run missing roles surfaces as a launch/start failure, roles are set via set_harness_agents (whose result/conflict payload reports the current assignment) or the Settings UI, and the readiness banner is a UI affordance on the harness page.
  • Human gates stay UI-first — and are opt-in per run (M11). A run's own agent token has no MCP path to approve its own gate, push, merge, or unfreeze its run. Freeze/resume are admin operator tools, never the run agent's. By default (requireHumanGates: false) an APPROVED review auto-resolves the gate it opened (authoring spec, decomposition PRE_EXECUTION, stage FINAL_ACCEPTANCE) server-side, as the run's creator with source automation — through the same audited resolveAgentApproval, so the coverage validation, the evidence gate, and the freeze predicate all still apply, and any failure reopens the gate PENDING for a human. Setting requireHumanGates: true parks every gate for a person.
  • Local commit authorization is a narrow, authorization-only tool. The external runner performs the local commit itself under a repo:commit permission check + SHA/diff report — the commit is not brokered. Because authorizeLocalCommit is internal today and a heartbeat/read call must never become a mutation boundary, H10.6 exposes that check as a dedicated authorization-only MCP/admin tool that returns the authorization without minting any credential. The subsequent push/PR still uses single-use brokered repo:push / repo:pr_create credentials, and merge stays the separate always-human M8 MERGE gate.
  • Auto-start is the run default since M11 (autoAdvanceStages: true), opt-out per run in Settings / set_harness_policy; the workspace-wide harness.stage_ready automation action remains available and composes with it (same guarded launch op, deduped).
  • UI scope. The harness page exposes agent configuration, authoring and decomposition starts, inline gate decisions, current-stage launch, policy settings, and freeze/resume/cancel through the same operations as MCP.

Current wiring. The run/stage spine, registry projection, artifacts, findings, freeze/cancel, commit gate, MCP read/write/admin tools, and agent-role configuration (H10.2 set_harness_agents), explicit start-authoring (H10.3 start_harness_authoring, DRAFT → AUTHORING plus the run-bound authoring job), the coverage-enforced decomposition path (H10.4 start_harness_decomposition + submit_plan's harnessStages proposal channel + the accept-gate mint of ordered stages/Work Orders, PLANNING → READY), launchable hand-authored stages (H10.5), and the harness-aware reference runner (H10.6 — the runner detects a stage job from get_agent_job's harness block, records stage + commit evidence, authorizes the local commit via authorize_harness_local_commit, and brokers single-use push/PR credentials, honoring freeze in the external process), and native next-stage orchestration + run completion (H10.7 — the opt-in harness.stage_readystart_harness_stage_run automation action launches the next stage deduped against an active stage job, and committing the final stage moves the run to COMPLETED atomically while emitting harness.completed) are live. The full spec→completion loop is therefore wired end to end. The operator UI (H10.8) closes the loop: an admin configures agents (at create time or in Settings), starts authoring/decomposition, approves/rejects the run's plan gates inline, launches the current stage, and freezes/resumes/cancels — all from the harness detail page, each calling the same ops the MCP/parity surfaces use. The Stages tab also shows the structured decomposition proposal a pending plan gate will mint, and get_harness's tool/doc text points MCP users at the companion calls for gate/run state and is explicit that agent-role readiness has no read tool (the output schema stays frozen). The critical UI paths are covered by Playwright with deterministic, provider-free mocked state; the actual agent loop stays covered by unit/integration smoke tests, since CI cannot run real Claude/Codex/git-provider sessions.

Runner concurrency and checkout isolation. Stages remain serial inside one run, but separate harness jobs can execute concurrently—even against the same repository. Each AgentJob owns a stable native Git worktree derived from the trusted STACKS_RUNNER_REPOS mapping and its opaque job id. The worktree survives review turns, gates, freeze, revisions, and runner restarts; later stages resolve their base from the preceding stage's branch tip so stacked, unmerged work is preserved.

STACKS_RUNNER_MAX_CONCURRENT_JOBS bounds agent turns and STACKS_RUNNER_MAX_CONCURRENT_COMMITS bounds commit continuations; both default to 1. STACKS_RUNNER_WORKTREE_ROOT optionally selects the worktree parent. Duplicate webhook/poll deliveries coalesce by job id before consuming a slot. Clean terminal worktrees are removed through Git after ignored bootstrap output is cleaned; source-dirty, frozen, gated, or unverifiable worktrees are retained for operator recovery and are never force-deleted automatically.

Local smoke test

The remediation is complete only when this scenario passes locally, with mocked agent turns, a local fake repo, and no production credentials or real CI — the same deterministic shape used for the rest of the harness tests:

  1. Seed a workspace with an anchor task, two active agent profiles, and linked planner/reviewer PATs.
  2. Create a harness (repo + branch policy + planner/reviewer roles).
  3. Start authoring, submit a PRD plan through the runner or a mocked MCP client, approve the gate, and observe AUTHORING → PLANNING.
  4. Start decomposition, submit covered proposed stages, approve the gate, and observe READY plus launchable stages with Work Orders.
  5. Start the first stage, run the harness-aware runner against the fake repo, record diff/test + commit evidence, approve required gates, and observe the stage become COMMITTED.
  6. Let harness.stage_ready launch the next stage, or show one clear manual CTA when auto mode is off.
  7. Commit the final stage and observe the run become COMPLETED with harness.completed.
  8. Confirm the Control Tower has no stale harness approval / stuck items for the run.

Reusable Runbooks

The kickoff prompts are not copy-pasted blocks. A Runbook is a versioned, checksummed HarnessPromptTemplate bound to a stage kind whose body references live Jentrix ids / MCP tools / state queries through {{placeholders}}. At launch the placeholders resolve against the concrete run + stage; the reviewer Runbook additionally interpolates the review-continuity block and a reconciliation instruction. A Runbook must be approved before it can be rendered (Codex reviews its safety/scope/turn-discipline first), and an approved Runbook with any unresolved placeholder fails closed rather than launching with raw {{...}} text. Runbook bodies are redacted on ingest — no raw secrets ever reach Postgres, a prompt, or the UI.

Harness templates (M18.3)

A Harness Template is a reusable, versioned pipeline definition: the ordered steps a run should have, the roles each step enables, its evidence requirements, and the run-level defaults a launch starts from. Templates live at Settings → Harness Templates; the ops core is src/server/harness-templates/.

A template is a generator, never a state machine. Instantiating one produces ordinary HarnessRun / HarnessStage rows, and the selected path's shipped convergence authority is preserved unchanged — the cron sweep plus the nextAction decision table on the unmanaged path, Temporal on a managedByTemporal run. Templates add no third state machine, no advance leg, and nothing in the sweep reads the two template tables (AGE-292 extension).

That is a tested property, not a design note. A template-launched run converges end to end through the sweep alone with every callback and webhook dropped, in the same number of ticks as a hand-created run (src/server/harness/liveness.sim.test.ts), and on a managedByTemporal run the sweep performs no advancement at all while the Temporal/outbox path converges or parks the run visibly (src/server/harness/managed-liveness.sim.test.ts).

A step whose roles have no eligible worker parks visibly. Publish and launch validation refuse an enabled role with no worker before a run exists, so the case that survives is a worker that stopped being eligible between launch and execution — deactivated, token revoked, role row replaced. The stage then moves to BLOCKED carrying the reason ("Stage N has no eligible worker — …"), which the harness page and the Control Tower show like any other block. Automation deliberately does not retry it: a relaunch would resolve the same roles and park again, spending the stage's shared retry budget for nothing. Assign an active planner/reviewer worker and the next sweep retries by itself — the park is re-derived from live state every five minutes, so there is nothing to clear by hand.

And a launched run actually starts. The launch mints the stages' Work Orders and fans out harness.stage_ready, the same signal the decomposition mint and each stage commit emit, so the first stage opens on the run's autoAdvanceStages policy — no template-specific dispatch path, no operator step in the happy path. On a managed run that first attempt is what pnpm harness:poll carries into Temporal. See Launching a template below for the full landing state, the two pre-attempt parks and the dispatcher.

Draft and published versions

HarnessTemplate is the stable identity. HarnessTemplateVersion holds the definition document, and its status is the whole contract:

StatusMeaning
DRAFTThe one mutable version. At most one per template — enforced by the partial unique index one_draft_per_template, so a concurrent fork cannot split the authoring head.
PUBLISHEDImmutable. Content-addressed by contentHash; this is what a launch pins.

A published version cannot be edited — an edit creates a new draft version. That is one rule with three enforcement points that must agree: assertVersionEditable (the ops-core refusal, a 409), planDraftEdit (the constructive half: fork version N+1 seeded from the published head, with basedOnVersionId lineage), and the partial unique index (the database backstop under a race). Saving an edit against a published head therefore succeeds — it lands in a new draft, and the UI says so before you type.

Validation gates publish. validateDefinition classifies findings as error or warning. Any error blocks publish and the refusal names each one; warnings never block (a template you cannot ship because it lacks a human gate just teaches operators to fake one). The validation summary an operator reads in the builder is produced by the same function that gates publish on the server — there is no client-side approximation.

Cloning forks a template, never a version. cloneHarnessTemplate copies one version's definition into a new template whose origin is CLONE and whose only version is a DRAFT — even when the source was published, because the copy is content nobody has approved yet. Both operator surfaces reach it: the registry row clones the published head (falling back to the draft when nothing is published), and the builder's status bar clones the version on screen by id, which is the only way to fork a superseded published version or the open draft deliberately. Both call the same action, so the admin check and the always-a-draft rule are the ops core's, once.

Promotion from a successful execution

promoteHarnessTemplateFromRun derives a draft template from a terminal, successful run's recorded stages. "Successful" is three conditions, all required:

unarchived  AND  status === COMPLETED  AND  acceptanceState === ACCEPTED

Both halves are load-bearing and they answer different questions. HarnessStatus.COMPLETED asks did every stage commit? — re-derived from the stage rows rather than trusted from the status column, since a template minted from a run whose stages did not actually commit would propagate a broken pipeline. acceptanceState (HAI S4) asks was the accepted spec satisfied? Treating the first as the second is the completion-mistaken-for-acceptance conflation, and it is refused. archivedAt is checked directly because status does not carry it — an archived run can still read COMPLETED.

ACCEPTED is an equality test, not an allowlist, so every other state fails closed: PENDING / WAITING_OPERATOR are verdicts still in flight, CONDITIONAL is a waiver that must never read as verified, and LEGACY_UNATTESTED means acceptance was never established at all — the absence of a verdict, not a passing one. An unrecognized state is refused too, the same way an unknown evidence tier ranks below every real one. (Adjudicated on finding cms1kjaqt009204laqbvgmrkc, 2026-07-27: widening this gate is a spec change, not an executor's call.)

Mapping: a COMMITTED stage becomes an active step; a SKIPPED stage becomes an archived step (the pipeline's shape is kept, the skipped work is not resurrected); evidence comes from the stage plan's verification commands, dropping to SELF_REPORTED when a stage recorded none; and managedByTemporal carries over, so promotion never migrates a run between the two convergence authorities.

Ineligible runs are refused with an explanation that distinguishes the case — archived (it was retired), not_terminal (wait for it), terminal_but_unsuccessful (it ended without succeeding), acceptance_unresolved (it committed, but acceptance is open, waived, or was never established), no_committed_stages (nothing to derive from). The picker on the templates page is fed by listPromotableRuns, which runs that same predicate, so the UI never offers a run the action would then refuse — and an empty picker says what would make a run eligible rather than rendering a dead control. A promoted draft is ORDINARY: it carries a normal validation summary and is publishable only through the same gate. Promotion buys provenance (promotedFromRunId), never a bypass.

Export and import

Export produces a portable stacks.harness-template document carrying credential references only — never raw secrets (M18-AC02). Three mechanisms, in descending order of how much you should trust them:

  1. The type makes a secret unrepresentable. A credential reference is { alias, kind, scopes, description }. There is no value/token/secret field to fill in, and the schema is .strict(), so one cannot be added by a caller.
  2. The exporter drops workspace-local binding. credentialRecordId names a CredentialRecord in this workspace and is meaningless elsewhere, so it is stripped; the importing operator rebinds by alias.
  3. The exporter scans and refuses. Free text (titles, contracts, verification commands) can carry anything someone pasted, so the whole document goes through the M4 redactor (src/lib/redact.ts — the one secret detector in the codebase) and the export fails on a match rather than emitting a silently-redacted document that would break at run time. The same guard runs at authoring time and on import.

Imports always land as a DRAFT: the document is untrusted input, its credential aliases are unbound, and publishing is the importing operator's decision.

Per-step worker assignments (R7)

A step declares which roles it enables (planner · executor · reviewer · validator) and, next to that, who runs each one. The resolution is pure and lives in src/server/harness/step-assignments.ts; it is the same function the builder previews with and the launch records from, so a preview cannot drift from what happens.

Resolution is total over enabled roles, and every row names its source. For each role a step enables, the resolver returns an entry — even one nothing assigned, which resolves as unassigned. Bindings are inherited per role and wholesale, in this order:

the step's own assignment  →  the template default  →  the run's role config  →  unassigned

The winning level is carried on every resolved row (source), and the builder renders it as a badge on each step. That is the PRD's hard rule: template-level defaults are allowed, but inheritance is never invisible — a default that silently bound a worker would be indistinguishable from a step nobody configured. Inheriting half a worker (a step's model on the template's connection) is deliberately impossible: a role block is taken whole, because a mixed triple is a pairing the operator never chose.

Order is declared, not derived. The primary is order 0 and fallbacks take 1..n in their declared array order — no scoring, no clock, no set iteration. A fallback chain with no primary is an error, and a chain that falls back onto the worker that just failed is a warning.

A launch resolves "active release" into a concrete immutable id. Each resolved assignment records the release the attempt runs (PINNED keeps its pin; ACTIVE_AT_LAUNCH resolves the worker's active release at launch and never follows the pointer afterwards), plus the runtime adapter, provider connection, model profile, and the narrowed permission revision (R5 — narrowing only: the M18.2 compiled bundle intersects, it never widens). Pin precedence is assignment → resolved release → worker profile: the release sits above the mutable profile fields, or "pinned" would be a suggestion.

The packaged self-hosted runner consumes that resolved row directly. Its setup config records the previewed role/adapter/connection/model tuple; each claim presents it, obtains a node-bound one-shot credential for a subscription connection, and requires the returned workerExecution to match exactly before starting Claude or Codex. Harness executor turns use the same path. The previous process-wide provider choice remains only as a noisy one-release compatibility path for manual legacy startup; jentrix runner up always uses durable bindings and never falls back.

A PINNED release never falls through to the worker profile, and that rule is worth stating separately because the failure it prevents looks like nothing. PINNED names an old, deliberately inactive release; the profile's runtime adapter and provider connection describe what that worker points at today. Had the pin's gaps been filled from the profile, the recorded row would pair release 3's id with release 9's adapter — a resolution that never happened, indelibly written down as though it had. So an unresolvable pin records nulls plus an unknown_release error, never a plausible substitute. Two dimensions are exempt because no release carries them: the model profile is a worker draft pin and the compiled permission revision is bound to the worker, so both still resolve from the profile under either mode. The catalog loader (loadAssignmentCatalog) collects pinned release ids from the assignment document, not from the loaded profiles — a release nobody currently points at is exactly what a pin is, and profile-derived loading can never see one.

Those rows are stage-scoped HarnessAgentRole rows written inside the launch transaction (writeStageAssignmentRows), so an attempt and its assignment record commit together. Run-level rows are the ones with harnessStageId: null and every run-level read filters on that, so a step's assignment can never be mistaken for the run's planner. Re-materializing archives the previous rows rather than deleting them, and the partial unique index HarnessAgentRole_stage_role_order_active is the concurrency backstop for one live row per (stage, role, order). The rows are a record, not a gate: an unresolvable pin lands as a null rather than blocking a launch that works today; refusing an unbound role is the template validation's job (bindingRequired), before a run exists.

Export strips every workspace-local id from an assignment (agent profile, release, connection, model profile, permission revision) exactly as it strips credentialRecordId, and degrades a PINNED mode to ACTIVE_AT_LAUNCH — what travels is the shape of the assignment and the chain's order; the importing operator rebinds the workers.

Validator execution semantics (R7)

A validator attempt runs read-only verification. It may read the workspace and repository and produce findings; it may not mutate either. The rule is pure (src/server/harness/validator-semantics.ts).

The guarantee is made at the claim, not at the boundaries. A claim is the one doorway where a turn is handed everything it could mutate with: the run row that authorizes it, the run-scoped bearer minted against that run, the credentials the broker will release to it, and — on the runner's side of that answer — the checkout it provisions for the leg. Nothing downstream can take a writable worktree back once it has been handed over, so "read-only" is a property of how an attempt was DISPATCHED and cannot be retrofitted by a later refusal.

So the question claimAgentJob asks is not "is this a validator?" but "will this dispatcher open the attempt read-only?" (validatorDispatchRefusal). A worker whose stamped schema AND app revision the control plane positively recognizes is running this build, so it carries the read-only lane below and its validator claim is authorized; an un-stamped, legacy or mismatched worker is refused, exactly as every worker was before the lane existed. The strict predicate is isRevisionAttested (src/lib/revision.ts) — deliberately narrower than the M12.3b compatibility gate, which is inert on an un-stamped control plane: a capability granted on the strength of a worker's code must never be granted on an unverifiable stamp, so "cannot tell" reads as "no".

A live validator attempt is a durable row, not an inference. AgentRunRole carries a fourth value, VALIDATOR (appended last — Postgres enums are append-only), and it is the whole hinge of the repository half. HarnessRoleKind.VALIDATOR describes a step's configuration; AgentRun.role = VALIDATOR is a running attempt. When the stage's per-step assignment resolves the claiming identity to the validator role, resolveClaimAttemptRole opens the run under it and claimAgentJob returns it — and that returned role is the single field the runner keys its checkout off. Without this value the lane below was correct, wired, and unreachable: expectedRunRoleForPhase answered EXECUTOR for every execution claim, so no production path could ever hand the runner a validator role to act on. Note what the attempt role deliberately does NOT change: the release preflight, the compiled scope-class check and the manifest pin all still key off the phase's role, because a validator takes the same turn with strictly less authority and must not skip a gate its executor claim passes.

runRoleSatisfiesPhase (workflows/state-machine.ts) bounds what that role may then do. It is a different question from expectedRunRoleForPhase — "is this run a legal turn-holder here?", not "which role does this phase dispatch?" — and a VALIDATOR run satisfies EXECUTING / EXECUTION_REVISION and nothing else. It must satisfy those, or the attempt could never close its own turn and its runner-attested evidence would have nowhere to land; it must satisfy no others, because authoring a plan or casting the review verdict that gates the stage are exactly the two turns a read-only role must never hold.

Refusing every validator claim outright would have been simpler, and it was wrong: it makes "cannot mutate" true only by making the criterion's second clause — their evidence lands via the runner-attested TEST_EVIDENCE path — unreachable. Half a guarantee. Two identities are still checked, because two can carry an assignment into one claim: the agent the JOB binds to the claimed role and the worker the claiming CREDENTIAL acts as; either one resolving to VALIDATOR is treated as a validator claim.

The runner's half: one checkout lane, keyed off the durable role. agents/lib/role-repo-access.ts maps the run role the claim returned to a checkout decision, and it is total and fail-closed: EXECUTOR → writable worktree (byte-for-byte the old behavior); PLANNER / REVIEWER → read-only; VALIDATOR and every role it does not recognize → read-only, adopt-only. driveHarnessStage — the path harness stage attempts actually take, not the generic M7 leg — resolves its worktree primitive from it: the read-only lane calls adoptExecutionCwd (which creates nothing) and parks when there is no tree to adopt, rather than degrading to a writable checkout or to the shared grounding checkout, whose tree lacks the stage's uncommitted change. The model session is sandboxed with the map's repoReadOnly — never a hand-written literal — through both the host and docker drivers.

The control plane cannot inspect a runner's filesystem, so that half is pinned by an attested test rather than by trust: validator-semantics.test.ts rides the stage's frozen verification commands and (a) imports the lane module across the dependency firewall — legal precisely because it has zero imports — and executes every case in-process, (b) asserts it equals REPO_ACCESS_BY_DISPATCH_ROLE here entry-for-entry including the fail-closed default, and (c) scans both dispatch paths' source to prove they still consume it. Drift or un-wiring is a red attested command. The behavioural half — driveHarnessStage genuinely never calling ensureExecutionCwd for a validator — runs in the agents package's own runner (agents/role-dispatch.smoke.test.ts), because that module imports the provider SDKs.

The remaining boundaries are defence in depth — each refuses an effect on the assumption that the dispatch guarantee above somehow did not hold:

BoundaryRefusal
claimAgentJob (dispatch — the guarantee)A validator claim is authorized only for a worker the control plane can positively identify as running the read-only lane; anything else is refused before a run is opened. When authorized, the run is opened as AgentRunRole.VALIDATOR and the claim returns that role, so the runner resolves a read-only, adopt-only checkout from it.
authorizeLocalCommit (the governed repo boundary)A worker holding the stage's VALIDATOR assignment cannot authorize a commit — refused server-side, not left to the runner's good behavior.
attachHarnessArtifact (evidence capture)A validator may author TEST_EVIDENCE and REVIEW_REPORT only. DIFF_OR_PATCH / EXECUTION_SUMMARY describe a change it never made, and unattested TEST_EVIDENCE is model prose wearing the name of evidence.
validateDefinition (authoring)A validator-only step carrying a committing commitPolicy is an error, so the contradiction surfaces while authoring rather than three days into a run.
safe() (the workspace boundary)Every non-read tool call by a governed worker credential acting inside an open VALIDATOR turn is refused unless it is the validator's own submission path. This is where "cannot mutate the workspace" is enforced, because the MCP surface is the one doorway a model's turn mutates a workspace through.
Tool surfacerecord_harness_acceptance_evidence is never a validator tool — operator-tier acceptance evidence is an ADMIN-only human attestation. authorize_harness_local_commit is refused for the same reason from the other direction.

The workspace boundary needs one more thing than the repository one, and it is the reason it lives at safe() rather than at the mint. A validator attempt's durable identity is AgentRun.role = VALIDATOR, but the model's turn does not present a bearer that names its run: agents/workflow-runner.ts hands the session the SHARED planner/reviewer role token, and only the runner's own submissions travel on a run-scoped bearer. Resolving the acting role from ApiToken.mintedForRunId alone would therefore govern the runner and leave the model's turn — the half that can actually mutate a workspace — ungoverned. So actingValidatorRunRole (src/server/workflows/operations.ts) resolves both: a run bearer by the run it names, and a shared role token through the job that hands it its turn (plannerTokenId/reviewerTokenId/executorTokenId) — the same binding assertTurnToken uses to decide whose turn it is. It asks only whether an open validator turn exists for that credential, deliberately not also requiring that run to be the job's activeRunId: a superseded-but-still-open validator run is precisely a case where writes should still be refused, so the narrower query would fail open.

Its evidence therefore arrives through the shipped runner-attested path: the runner executes the stage's declared verification commands and attests command, exit code and output digest against the reviewed diff. Nothing about the acceptance ladder is special-cased for validators — and nothing needs to be, because that path never went through the model in the first place.

"Read-only" deliberately does not mean "no write scope class": v1 has three scope classes and every claimed turn needs write just to submit its own result (TURN_SUBMIT_SCOPE_CLASS). It means the validator may write its own evidence and findings and nothing else, which is what the boundaries above enforce. Stripping write from a validator's credential would buy "cannot mutate" by making the validator unable to land the runner-attested TEST_EVIDENCE the same criterion requires — so the narrowing happens one layer in, at the tool: VALIDATOR_SELF_SUBMISSION_TOOLS (submit_harness_artifact, submit_harness_stage_result, submit_execution_result, update_harness_finding) is the whole write surface a validator keeps, derived from the turn protocol the runner actually drives. What the credential ceiling (runRoleScopeCeiling) does buy is narrower and still real: admin is structurally unavailable to any run bearer, and a run role no ceiling names mints nothing at all rather than inheriting a read+write default. The acting identity comes from mcpActor().agentProfileId matched against the stage's assignment rows; a caller with no assignment (a human operator, the UI) resolves to null and every shipped path behaves exactly as before. A worker holding two roles on one stage resolves most-restrictively to VALIDATOR — the M4 rule, for the same reason, and roleForAnyActor extends it across identities so the permissive one never rescues the restricted one.

One authoring consequence follows from that, and it is deliberate: binding the same worker to a step's executor AND validator roles makes its claim a validator claim, so that step's work runs read-only and never commits. Bind a different worker to the validator role. A worker asked to verify its own output was never the point of the role.

Launch: the review, the freeze, and dry-run mode (§10.4)

Launching a template resolves everything the run will execute under, shows it to the operator, and then freezes exactly what was shown.

One function produces the preview and the freeze. buildLaunchPreview (src/server/harness-templates/launch-preview.ts, pure) builds the launch resolution from durable facts; launchHarnessTemplate stores that value VERBATIM in HarnessRun.launchSnapshot with its sha256 in launchChecksum. There is deliberately no second snapshot builder on the write path — that is where a preview and a freeze drift apart — so "the preview matches the frozen resolution byte-for-byte" is a canonicalJson string comparison, not a convention. The snapshot is total over eight sections and carries no clock, no run id and no "resolved at": a snapshot with a timestamp in it could never be compared with the preview that produced it.

SectionWhat is frozen
templateTemplate + published version id, versionNo, contentHash, active step keys
releasesPer step/role/fallback-order: worker, the concrete immutable AgentRelease, release mode, runtime adapter, provider connection, model profile
permissionsThe narrowed permission revision per assignment, its compiled bundle hash, and the compiled-artifact verdict (VERIFIED, or the refusal code)
policyThe run's stage policy + every enabled M4 policy's currentRevisionId (recorded, never evaluated — assertPolicyAllows at the action boundary is still the only evaluator)
credentialsCredential references (alias, kind, scopes, record binding, requesting steps) — never material
budgetmaxSpendUsd / maxDurationMinutes / maxTurns / maxToolCalls
gatesPer-step human gates + the run-level requireHumanGates
repositoryrepo, base branch, branch template, branch strategy, allowed paths

Blockers ARE the refusal. The preview's blockers (no published version, an enabled role with no worker, an unbound or unusable credential, a committing pipeline with no repository, an archived template) are what launchHarnessTemplate refuses on, so the button an operator sees and the ops core's 409 can never disagree. Warnings never block.

Compiled permission artifacts are a blocker, decided by the shipped gate. Every bound assignment must resolve a permission revision whose compiled bundle is present, complete and current — assertCompiledArtifactsForLaunch (src/server/permissions/compilation.ts), the same function the claim gate runs, called directly from the pure preview. Five ways to fail, each a distinct code recorded per assignment in permissions[].compiledArtifactStatus: permission_revision_unset (the worker has no published revision), permission_revision_unresolved (the revision id names no row here), permission_artifacts_absent, permission_artifacts_incomplete, permission_artifacts_stale (compiled from content the revision no longer holds), permission_compiler_stale (an older compiler than this deployment runs). The freeze records VERIFIED for the rest, so an auditor reads the verdict off the snapshot instead of inferring it from a blocker nobody kept. The fact loader reads compiledArtifacts + content, not just the hash — a loader that fetched only the hash made "stale" unrepresentable.

The operator surface is the Launch review on the harness-template settings page (src/components/harness-templates/launch-review.tsx): resolve, read the eight groups plus missing prerequisites, warnings and the resolution checksum, then launch live or as a dry run. It renders the preview and computes nothing about it — the section list is LAUNCH_SNAPSHOT_SECTIONS and the renderer is a total Record over that union, so a group added to the snapshot must be rendered or the build fails. Editing any launch input discards the open review, because the checksum binds a launch to the resolution the operator actually read.

expectedChecksum binds review to launch. Pass the reviewed checksum and a launch whose resolution has since moved — a re-pointed active release, a republished permission revision, an edited policy — fails with CONFLICT instead of freezing something nobody approved. It is expectedUpdatedAt's idea applied to a resolution rather than a row, and optional, because the scripted CI client legitimately launches without a human review.

Launching writes the run, its stages (in template order), the stage-scoped HarnessAgentRole rows from the same resolution, one Work Order per active step, the dry-run projection and both audit rows (HARNESS_LAUNCHED, on the run and on the template) in ONE transaction.

A live, task-anchored launch produces a READY, Work-Ordered run and starts it. The Work Orders come from the shipped per-stage mint the decomposition path uses, the first-undone-stage cache is recomputed by the shipped writer, and the run is created READY — the same state decomposition leaves a run in. The launch then fans out harness.stage_ready, exactly as the decomposition mint and each stage commit do, so the first stage starts on the run's autoAdvanceStages policy like any other run's does. Nothing template-specific dispatches work: the launch reuses the shipped event and the shipped auto-advance actuator, which is what "a generator, never a state machine" means in practice.

Dry-run and anchorless launches stay inert. A dry run mints no Work Order, fans nothing out and starts nothing; a launch with no anchor task does the same, because a Work Order requires a task. Both land the run in DRAFT with non-executable stages — inert for the sweep, and a visible operator queue rather than a hang. The reason is recorded on the run's own HARNESS_LAUNCHED audit row (executable: false plus inertReason), so "nothing started" is something you read rather than infer.

Two things can stop a launched stage before it opens an attempt, and both park the stage BLOCKED with the reason on it rather than throwing into a log nobody reads:

ParkWhenWhat clears it
no_eligible_workerthe stage's roles resolve to no active worker with a usable tokenassign or reactivate one — on an unmanaged run the next sweep retries by itself, because the park is re-derived from live state
incomplete_dispatch_contexta managed run whose repository identity is missing or not a well-formed owner/nameset the run's repo; a managed run is not swept, so an operator restarts the stage

Neither is ever auto-retried: a relaunch would reach the identical refusal and spend the stage's shared retry budget reproducing it.

Per-stage worker binding. A stage attempt resolves its planner/reviewer from the CURRENT stage's frozen assignment rows — the primary first, then the recorded fallbacks in order — and falls back to the run-level rows when the stage has none, which is why non-template runs behave exactly as before. VALIDATOR rows are excluded by construction: a validator turn is read-only and can never become the acting planner.

Dispatching a managed run. pnpm harness:poll is the automatic dispatcher: it reads the durable AgentJob rows of managed runs and starts one Temporal stage workflow per attempt, with workflowId = jobId so a re-poll lands the same single workflow. The start is issued under Temporal's REJECT_DUPLICATE workflow-id reuse policy, so the id is refused for its whole life — including after that workflow has CLOSED. That matters because the poller re-reads the same durable row every interval: under the SDK's default (ALLOW_DUPLICATE) a poll after the workflow finished would start a second workflow for the same attempt, which would make the dispatcher a re-execution authority instead of a dispatcher. pnpm harness:stage remains the manual, one-attempt dispatcher. The dispatcher writes no domain row in any branch — an attempt it cannot complete (a pre-gate row with a malformed repo, say) is logged and skipped, never parked; parking from there would be a third authority. A deployment that runs neither command behaves exactly as it did before, and its managed work waits for an operator: the sweep is fenced out of managed runs by design, which is a topology choice the same way not running the outbox bridge is, not a bug in the run.

Dry-run mode

A dry run instantiates the same records with launchMode: "dry_run". Planning and review legs execute normally; execution legs are projected, not invoked with delivery authority. The decisions are pure (src/server/harness-templates/dry-run.ts); launchModeForStage / launchModeForAgentRun / actingDryRunLaunchMode (src/server/harness/launch-mode.ts) are the only readers of the column.

Inertness comes from the claim, like the validator lane's does. A claim hands out the run's authority and the runner's writable checkout together, so a dry run's EXECUTING / EXECUTION_REVISION / MERGE_GATE / DEPLOY_GATE claim is refused outright (dryRunDispatchRefusal) — the attempt is never opened, rather than opened and then policed. Everything below is defence in depth:

SurfaceBoundaryRefusal
branch · commit · PR (local)authorizeLocalCommitdryRunRepositoryRefusal
recording a commit SHArecordCommitEvidencedryRunRepositoryRefusal
crossing / executing the commit boundarycrossCommitBoundary, executeCommitBoundarydryRunRepositoryRefusal
merge · production deploydispatchGateExecutiondryRunDeliveryRefusal
workspace mutation (MCP tool call)safe() (MCP)dryRunWorkspaceMutationRefusal
outbound messagesendOutbounddryRunOutboundRefusal
issuing a credential grantissueCredentialGrant, brokerRepoToken, brokerCloudTokendryRunCredentialRefusal
redeeming credential materialresolveGrantForExecutiondryRunCredentialRefusal

Three of those rows exist because the obvious single boundary is not enough. Recording a commit SHA matters as much as authorizing one: recorded evidence is what wakes advanceCommitBoundary, so a dry run that could file a SHA would have a live path from "projected" to a real push/PR decision. Issuing a credential grant matters as much as redeeming one: the CredentialGrant row IS the platform's authorization record — what the Control Tower renders and what an auditor reads as "this run was authorized for deploy:production" — so refusing only at the mint leaves that record standing. And the outbound send is refused inside sendOutbound rather than only at the MCP tool, because sendOutbound is also a UI server action, and refusing before the enqueue means a dry run leaves no PENDING ConnectorDelivery for the sweep drain to pick up later. The two commit-boundary rows are sweep/outbox-driven, so they decline with an inert outcome instead of throwing into a drain — but they write a guard-refusal breadcrumb first, because a boundary that declines silently reads exactly like one that had nothing to do.

Where in the function matters, not just which function. The two direct brokers (brokerRepoToken, brokerCloudToken) find-or-create the workspace's durable GITHUB_APP / VERCEL CredentialRecord before any grant exists, so a refusal that lands only inside the grant transaction still lets a dry run write a real, admin-visible credential row into the workspace — an unprojected mutation performed by a run whose entire contract is that it produces none. Both therefore assert first, before the provider-token read and before that find-or-create, and re-assert inside the grant transaction to close the same TOCTOU the freeze guard closes. The rule generalizes: for a boundary that mutates before it authorizes, "the function refuses" is not the property — "the function refuses before its first write" is.

EXTERNAL_EFFECT_SURFACES in dry-run.ts names the refusing boundary for each surface, and dry-run.test.ts resolves every name against the imported production function it drives. That check exists because the table named executeGatedDeliveryAction — a function that has never existed — while every assertion over it passed. A surface table nothing verifies is documentation, not a contract.

The MCP gate is deliberately not nested inside the compiled-permission session: a model's turn presents the SHARED role token (agents/workflow-runner.ts), which names neither a worker nor a run, so actingDryRunLaunchMode resolves it through the job's turn binding — and gating that on an unrelated plane's state made the one credential shape a dry-run planning leg actually uses skip the check.

A dry-run leg keeps every read tool and its own submission path (so its plan, review, findings and evidence can land) and nothing else — the VALIDATOR_SELF_SUBMISSION_TOOLS precedent, because every claimed turn needs the write class just to close itself.

The projection is a SimulationRun, and its plan is EMPTY. The launch links one (HarnessRun.simulationRunId) carrying the inert intent records in projectedEffects — one per step, plus the credential grants and human gates a live launch would have required. plan: [] is the load-bearing part: executeSimulation iterates the plan, so there is literally nothing to execute, whoever calls it and whatever its status. Inertness is a property of the data, not of a guard someone has to remember. The linkage and the intent records render in the execution's Dry Run tab.

Dry runs never satisfy acceptance for a live launch. evaluateHarnessAcceptance admits no evidence from a dry run (admitsAcceptanceEvidence), so every AUTOMATED criterion stays PENDING and every OPERATOR criterion stays WAITING_OPERATOR with a problem naming the mode — a dry run cannot reach ACCEPTED. recordAcceptanceEvidence refuses at the door rather than storing a human attestation about work that never ran, and assessPromotion refuses a dry run with its own reason (dry_run) instead of the misleading "acceptance is still pending".

launchMode is a code-validated open string, and isDryRun is an EXACT match: an absent or unrecognized value reads as live, because silently de-authorizing a real execution on a corrupt row is a worse failure than the one a fail-closed reflex would prevent. The launch boundary validates the mode with Zod, so only the two values are ever written.

MCP tools and the scripted CI client (R8)

Everything above is reachable from an agent or a CI job, not just the builder. Fourteen tools ship the same operations the UI drives — six reads (list_harness_templates, get_harness_template, validate_harness_template, compare_harness_template_versions, export_harness_template, preview_harness_template_launch) and eight ADMIN mutations (create_harness_template, update_harness_template_draft, publish_harness_template_version, clone_harness_template, set_harness_template_archived, import_harness_template, promote_harness_template_from_run, launch_harness_template). Archive and restore are one setter with an archived flag, the manage_labels idiom.

They wrap the shared ops core; they do not reimplement it. previewHarnessTemplateLaunch and launchHarnessTemplate are exported from the same operations.ts the authoring actions use, so the admin role gate, the published-version immutability rule, the publish validation gate, the secret-material refusal, the promotion predicate, the expectedUpdatedAt CAS and the launch freeze behave identically whichever surface you came through. A dry_run launch over MCP instantiates exactly the inert run and linked SimulationRun described above.

Two rules are stricter over MCP than in the UI: every create-style tool requires idempotencyKey, and every update-style tool requires expectedUpdatedAt. A form post is a single human intent holding a session; a CI job retries, and two pipelines edit the same template without seeing each other. Errors use the standard envelope, so a stale expectedUpdatedAt comes back as CONFLICT with the template's current state under error.current, and a launch whose resolution moved since the expectedChecksum you reviewed comes back as CONFLICT rather than freezing something unreviewed.

The CI client is agents/scripts/harness-template.ts (also wired as pnpm --dir agents harness-template):

STACKS_TOKEN=tm_… npx tsx agents/scripts/harness-template.ts \
  preview --template ht_123 --repo acme/app
STACKS_TOKEN=tm_… npx tsx agents/scripts/harness-template.ts \
  launch --template ht_123 --repo acme/app --dry-run \
  --expected-checksum "$CHECKSUM" --idempotency-key "$GITHUB_RUN_ID"

Commands: list, get, validate, publish, export, import, preview, launch. It is a thin shell — one tools/call per command, --json for a downstream jq. Authentication comes from STACKS_TOKEN; there is no --token flag, and an argument that looks like a provider key is refused before anything is sent, because an argument lands in shell history and the CI log before any server sees it. A template references credentials by ALIAS. Exit codes are 0 success, 1 server refusal (the envelope is printed verbatim to stderr), 2 bad invocation, and 3 "the call succeeded and the answer was no" — validate found errors, or preview found blockers — so validate && publish cannot publish what validation refused.

Scope

M18.3's stages ship the aggregate, versions, ops core, UI authoring, promotion, the per-step assignments + validator semantics above, the launch preview / freeze / dry-run mode, and the MCP + CI surfaces in this section. Template-launched liveness coverage is the remaining sibling stage of the same milestone.

Every failed or cancelled execution carries a terminal disposition and a next-action history (M18-AC21; R12's "every failed terminal state records a next action, disposition, or authorized cancellation"). The disposition lives on the run — HarnessRun.disposition / dispositionNote / dispositionAt / dispositionById, an OPEN STRING from HARNESS_DISPOSITIONS (RETRIED, REPLACED, ABANDONED, CANCELLED, ESCALATED, RESOLVED_EXTERNALLY) — and the history is append-only HARNESS_NEXT_ACTION_RECORDED activity rows, so "what did we decide, and what did we do?" is answerable from durable state rather than from prose.

The invariant is upheld by writers, not by a report:

  • Entering a terminal state records the disposition, in the same transaction. One pure table (TERMINAL_STATUS_DISPOSITION) maps each terminal status to what the platform records — CANCELLED → "CANCELLED" with a cancel_execution history row (cancelHarness), and BLOCKED → "ESCALATED" with an escalate_to_owner row (recordFailureDisposition, on the resume path that can land a run in BLOCKED). Neither value is a guess about intent: a run parked in BLOCKED is the platform handing the execution to a human, which is verbatim what ESCALATED means. Both writers guard on disposition: null, so an operator's recorded decision is never overwritten — and the history row is appended either way, because the history records what happened, not what the columns currently say.
  • The operator path cannot leave a failure undisposed. recordHarnessDisposition refuses (400) an action on a failed or cancelled execution that carries no disposition unless one is supplied. Once a disposition IS on record, an action may be recorded without changing it — "supplied the missing evidence" does not re-decide how the execution ended.
  • The sweep converges the rest. convergeUndisposedExecutions records the status's own mapped disposition (as SYSTEM, actorId null) for any failed or cancelled run that predates those writers or was left half-written by a crash. Idempotent, guarded on disposition: null, and it never overwrites a human's decision. Zero in steady state.
  • A missing disposition is still a VISIBLE debt, never a status gate. A BLOCKED/CANCELLED run with no disposition yet shows "Disposition required" on the execution page; it does not hold HarnessStatus anywhere. Gating a status on a human note would break the AGE-292 convergence obligation — the sweep must always reach COMMITTED or a visibly parked state.

§10.6 offers only state-legal actions. src/server/harness/dispositions.ts computes the legal subset of the PRD's twelve resolution actions, and every predicate rides a SHIPPED edge rather than a new table: pause_execution / cancel_execution are legal exactly when canTransitionRun allows FROZEN / CANCELLED; retry_attempt needs the stage's BLOCKED → READY edge (which opens a NEW attempt with lineage — a retry never mutates the historical one); approve_gate / reject_gate need gateForStageStatus to name a gate; edit_draft_template needs an unarchived template. So a change to the harness state machine moves this list with it. The rule is enforced server-siderecordHarnessDisposition refuses an illegal action with the legal set named, because a hidden button is not enforcement.

Archiving an execution (M18.4 R13)

An execution archives only from a terminal status (COMPLETED, CANCELLED, ARCHIVED). Archiving used to bypass the transition table entirely, which meant an EXECUTING run could be archived out from under a claimed attempt; archiveHarness now runs the shared archive preflight (src/server/archive/), which refuses a non-terminal run and names cancellation as the way through, and records the dependency report in the HARNESS_ARCHIVED audit payload either way. The same preflight governs the provider connections, model profiles, AI workers, permission templates, and harness templates an execution depends on — see Archive and restore (R13).

Human gates

Gate mode is per-run policy (M11). With the default requireHumanGates: false, gates opened by an APPROVED review (authoring spec, decomposition PRE_EXECUTION, stage FINAL_ACCEPTANCE) resolve automatically server-side — through the same audited resolveAgentApproval, with the coverage/evidence/freeze guards intact; a guard failure reopens the gate for a human. Set requireHumanGates: true (Settings tab / set_harness_policy) to park every gate for a person. Policy-driven gates (the commit gate under a REQUIRE_APPROVAL repo policy, merge, deploy) are governance, not ceremony — they always park regardless of this setting.

The post-gate chain is durable (AGE-295). Approving a stage's FINAL_ACCEPTANCE used to dispatch the two-step completion chain (validate_harness_stagecommit_harness_stage) via a best-effort serverless callback — a dropped callback stranded the stage invisibly (the AGE-291 incident class). The chain is now persisted to a GateSideEffect outbox in the same transaction as the gate flip, drained immediately post-response (the happy path keeps its latency) and re-drained by the cron sweep with retries and exponential backoff until each step succeeds — so losing the callback costs at most one sweep interval, never the run. A step that exhausts its retry budget goes DEAD (its chain cascades) and surfaces in the STUCK_HARNESSES queue with the failing step, attempt count, and last error named. The sweep's reconcile legs (below) remain as the safety net behind the outbox.

The sweep converges from one decision table (AGE-296). The reconcile sweep no longer decides inside hand-written legs: every candidate it collects (pending gates, approved-but-stranded acceptances, stalled commit-gate parks, blocked stages, launchable runs) is reduced to a loop snapshot and passed through one pure nextAction(state) table (src/server/harness/advance-decision.ts) whose actions dispatch through the same funnels as the live path. Candidate collection keyset-paginates inside its reconcile window wherever actionability can only be decided after enrichment (the stalled commit-gate leg), so a flood of newer non-actionable parks can never permanently crowd an older actionable stage out of a fixed-size page. The table is property-tested over the entire enumerated state space — totality (every state maps to an action), the never-actuate fences (Temporal-managed and frozen runs are observed, never driven; COMMIT/MERGE/DEPLOY gates are never auto-resolved; retries never exceed the shared budget or bypass a decision/round-cap park), and a pinned action fingerprint — so growing the stage machine without teaching the sweep is a failing test, not a stuck run. The enforced invariant: the cron sweep alone converges every stage to COMMITTED or a visibly human-parked state; after() callbacks and webhooks are latency optimizations only (liveness.sim.test.ts proves it with every side effect dropped, for template-launched runs as well as hand-created ones).

A park a retry cannot possibly clear is never retried into: an unattestable evidence tier and a stage with no eligible worker both park for a human rather than burning the stage's shared retry budget reproducing themselves. Both are re-derived from live state on every sweep rather than stored as a flag, so fixing the cause is the entire fix — the next sweep simply stops refusing.

Two authorities, never three. managedByTemporal picks which one owns a run: the sweep above, or Temporal through the transactional outbox. On a managed run the sweep is an observer that performs no advancement at all — not a gate resolution, not a retry, not so much as the breadcrumb note it writes before auto-resolving — while the Temporal/outbox path converges the run or parks it visibly, with the projection's canonical event key making a redelivered transition a no-op instead of a duplicate. Both halves are pinned together in managed-liveness.sim.test.ts, which drives the same world through both and audits the sweep's every dispatch.

Leases gate claiming, never recording (AGE-297). A WorkLease pin is a liveness artifact: it legitimately decays whenever a job parks at a human gate — heartbeats stop, TTL/orphan sweeps release it — so lease identity may gate claiming new work (launching an attempt, acquiring the stage), but must never gate recording the results of work already done. That mismatch is what stranded AGE-291a for 18 hours: validation refused a RELEASED pin on the very attempt that had finished the work. Every recording path — stage validation, parking a failed attempt, commit evidence, stage results, the M7 submit_* ops — now fences on durable attempt identity instead (the acting job must be the stage's newest non-archived attempt; a run must still be the job's activeRunId), with two deliberate edges: a TAKEN_OVER lease stays a hard supersession marker (an operator takeover refuses even the newest attempt), and a lapsed heartbeat TTL never rejects a submit (between TTL-lapse and the sweep's expiry action, the attempt is still legitimately current — refusing there rejected exactly the recordings that were safe).

The attempt fence is atomic, not check-then-write. A pre-write newest-attempt check alone is raceable — a retry can mint a newer attempt between the check and the write, letting a stale submission contaminate the stage-scoped evidence the validation gate trusts, bury the current attempt's commit evidence under a stale marker, or let a superseded run verify a finding. Every attempt-fenced write therefore re-asserts inside its own transaction via assertCurrentAttemptInTx (src/server/harness/operations.ts): it takes a FOR UPDATE lock on the stage row (no touch — updatedAt is load-bearing "parked since" state), then re-reads the newest non-archived attempt under that lock. The stage-execution launch transaction CASes the same row when it creates the attempt job, so under the lock the attempt set is frozen: a concurrent mint either committed first (the re-read sees it and the write aborts as stale) or blocks until the recording commits — the same legal order as a completed submission followed by a retry. This fence guards the execution-summary/evidence artifact writes and the branch pin (submitStageResultattachHarnessArtifact), the commit-evidence marker (recordCommitEvidence), and finding verification (resolveFinding → VERIFIED). Lock ordering invariant: every such transaction takes the STAGE row first, the pinned LEASE row (assertLeaseNotTakenOverInTx) second — the same stage → lease order the launch transaction writes in, so record paths can never AB-BA deadlock an attempt mint.

No silent no-ops (AGE-298). A guard that halts forward progress on a non-terminal stage inside a side-effect/reconcile path records why as a guard_refusal breadcrumb on the stage's activity (system-sourced, bounded, best-effort, deduped per attempt+guard pair; the no-progress detector ignores these rows so a refusal loop can't reset the stall alarm) — console.error on serverless is not observability. This covers the completion bridge's validation/park fences and the commit boundary's evidence rejections (unbound, stale-attempt, and taken-over reports — including the in- transaction races: a retry or takeover landing mid-record drops with the same breadcrumb, never as a swallowed caller-side log line). The recorded reason is rendered to humans: the STUCK_HARNESSES impossible-state and no-progress items embed the stage's newest refusal ("Last guard refusal (…): …") in their reason text — exactly where operators triage stuck stages. On the runner side, the commit poller has a deadman: when its poll budget exhausts, it re-reads freeze and stage status fresh (a stage that committed or blocked during the final sleep never false-alarms), and if durable state shows the impossible pairing (job DONE, stage not COMMITTED, no actionable pending gate — the runner mirrors the gate-visibility.ts predicate, so a stale PENDING FINAL_ACCEPTANCE row on a DONE job suppresses nothing) it emits a distinct [harness-deadman] ERROR marker line (monitor-alertable) and posts a comment on the run's anchor task — the marker once per stage per process, the comment retried until it actually posts — instead of reporting the catch-all "gated" forever.

Every parked gate is UI-first (the Control Tower / harness page). A run's own agent token has no MCP path to approve its own gate, push, merge, or unfreeze its run — those are human/operator decisions. (Freeze and resume are exposed as admin MCP tools for operators below, never for the run's own agent.) The harness detail page surfaces a run's pending plan gates inline and an admin can approve/reject them there directly — it resolves through the same Control Tower path (HARNESS_APPROVALSresolveAgentApproval), so there is one audited resolution, never a harness-specific approval shortcut. A frozen run's gate renders read-only (resume first); approve/reject is offered only while the run is live and unfrozen.

  • Spec / plan acceptance — approved at the authoring/decomposition plan gate.
  • Stage launch — the harness:advance policy governs the outer loop's advance boundary (launching the run's next stage): DENY blocks every launch; REQUIRE_APPROVAL parks automation/MCP launches so only a human "Start current stage" on the harness page advances the run (the UI click is the approval). Resolved purely (no persisted evaluation or generic approval item — the harness page and stuck queue are the human surface); with no policy configured the outer loop runs ungoverned (ALLOW), matching the commit boundary's default.
  • Commit gate — when push/PR policy returns REQUIRE_APPROVAL, a COMMIT approval is opened on the stage's COMMIT_GATE and parked in the Harness-approvals queue. ALLOW auto-commits; DENY blocks the stage. The local commit itself is runner-performed under a repo:commit permission check + SHA/diff report (not brokered); the push/PR uses a single-use brokered repo:push / repo:pr_create credential. Merge stays the separate, always- human M8 MERGE gate — merge is never folded into the commit gate.
  • Freeze / resume — a run-level freeze is the operator's stop button. It is enforced at every Jentrix boundary by one predicate: claim_* / submit_* reject or park, heartbeat_* returns frozen: true (the runner pauses on its next tick), the credential broker fails closed, and the artifact choke point rejects. Resume restores the exact recorded prior status and is attributed and audited.

Acceptance integrity (HAI)

The harness used to govern whether evidence objects and approvals existed, not whether they proved the original requirement — executor prose was recorded as test evidence, late-discovered violations could not block a review, and the final stage commit was read as product acceptance. The Harness Acceptance Integrity layer closes that gap with three mechanisms.

COMPLETED = stages committed; ACCEPTED = baseline verified. The run status keeps its H10.7 meaning (every stage crossed its commit gate — the UI now labels it "stages complete"). A separate HarnessRun.acceptanceState answers "is the accepted spec satisfied?": LEGACY_UNATTESTED (pre-HAI run, or a plan that mapped no criterion evidence — never silently accepted) → PENDINGWAITING_OPERATORCONDITIONAL | ACCEPTED. The policy is minted ONCE at decomposition acceptance from the pinned SpecVersion.definitionOfDone consolidated with the stages' criterionEvidence mapping (each criterion exactly once across the run, verbatim strings, AUTOMATED ⇒ ≥1 verification command — an invalid mapping blocks the plan gate). The cron sweep is the authoritative recalculation (a pure derivation over durable state — it never drives the stage machine); in-flow recalcs are latency optimizations, and every state change fans out harness.acceptance_updated. get_harness (detailed) returns the criterion matrix; a COMPLETED-but-unaccepted run surfaces in the HARNESS_MISSING_EVIDENCE Control Tower queue. Operator criteria are closed with the ADMIN tool record_harness_acceptance_evidence (verbatim criterion, audited, transactional recalc) — automated criteria can never be closed by hand.

Incident completion coupling (M18.4 R11). The same ladder carries the corrective loop: an execution correlated to an incident with a REQUIRED, undisposed follow-up sits at PENDING, and one whose follow-up an admin WAIVED is capped at CONDITIONAL — a waiver is a visible conditional outcome, never a clean success. HarnessStatus is untouched (a run parks on its acceptance axis, never its status axis, so the sweep still converges every stage). The waiver is the IncidentTask row's own admin-only, reason-required WAIVED disposition — set from the incident page, deliberately with no MCP tool, and permanent, so the verdict cannot lapse back to ACCEPTED. Which incidents belong to a run is resolved by the SHARED ownership rule in src/server/control-tower/lifecycle-correlation.ts (the one Control Tower's lifecycle view uses), and the read is fail-closed: a failed load throws and leaves acceptanceState unchanged rather than reading as "nothing outstanding". get_harness reports both counts (unresolvedRequiredCount / waivedRequiredCount) beside the finding counts.

Mandatory harness-flow gate. Every executor job runs one extra verification command, last, on top of whatever the stage declares: pnpm test:harness-flow (HARNESS_FLOW_GATE_COMMAND in agents/lib/harness-runner.ts) — sweep-only liveness, managed-path liveness (the sweep's no-advancement audit), decision-table totality, and the review-round/park accounting. The harness drives itself with the repo it is changing, so a stage can break the machinery carrying it and the symptom surfaces rounds later as an unattributable wedge (M18 stage 6 burned five review rounds on a round counter that never reset). It rides the normal verification path deliberately: real exit code, runner attestation, and a red gate landing in risks + testsNotRun where the validation gate and the reviewer both see it. Seconds, not minutes — the Playwright e2e suite provisions a database container per run and stays on the stage's own command list where a stage needs it.

Project-context injection. Each role turn is handed the repo's own contract plus the work it cannot see (agents/lib/project-context.ts): CLAUDE.md named as the standard its review is judged against (AGENTS.md symlinks to it for the Codex legs; the Claude legs load it natively via settingSources: ['project']), the base-branch commits its checkout does NOT have, and a git-truth line — HEAD, the uncommitted-change count, and how many commits the checkout trails the base branch by — measured in the checkout rather than remembered from a prior round.

Executor hooks (Claude legs). Two role contracts that were prose only are enforced mechanically in the SDK's query() hooks (agents/lib/executor-hooks.ts), on harness stage executor legs: a submission the RUNNER owns (submit_execution_result, submit_harness_stage_result, submit_harness_artifact for diff/test/summary kinds) is denied while the worktree has uncommitted changes, with the dirty paths named, so the ledger never receives a claim about a tree nothing has captured; and a turn trying to end with background work still running is blocked with a reason — twice, then allowed, because an unbounded block against a confused model is a new wedge and the park checkpoint is the backstop. Both are scoped deliberately: a generic M7 execution leg submits its own result from a legitimately dirty checkout, and ending with a dirty tree is the correct harness executor ending (the runner captures the diff next). Codex legs cannot be hooked — their honesty is enforced server-side by the attested evidence gate and the review-disposition rules. A stage worktree is cut from its base ref once and never rebased, so parallel work on the same repo — harness fixes landing while a PRD is implemented — is otherwise invisible to planner, executor, and reviewer alike. Every git read there is best-effort: a context lookup never costs a turn.

Runner-attested test evidence. The executor can no longer author TEST_EVIDENCE: the stage plan declares verificationCommands, the RUNNER process executes them itself in an ephemeral verifier (foreground, per-command timeout, explicit non-secret environment, and denied egress) and submits each captured output with an attestation — command, exit code, timeout flag, output hashes, evidence tier, and a subjectDigest binding it to the exact reviewed diff. Diff capture includes non-ignored untracked files without leaving them staged, so a newly-created test cannot execute outside the attested subject. The submit path verifies the attesting identity server-side: only the token that claimed the active attempt (AgentRun.claimedByRunnerTokenId) may attest, the attempt is fenced on durable identity (jobId + activeRunId, re-asserted in-transaction), and the claimed tier is capped by the token's ApiToken.maxEvidenceTier ceiling (null ⇒ INTEGRATION_LOCAL). Self-hosted bootstrap grants mint only INTEGRATION_LOCAL (D1, 2026-07: the local PROD_TOPOLOGY_LOCAL kind/Calico/gVisor lane was deleted — the next real tier is MANAGED_STAGING, a managed substrate, never a laptop cluster). The deployment-wide attestable ceiling is maxAttestableTier() (STACKS_MAX_ATTESTABLE_TIER, default INTEGRATION_LOCAL), and it is enforced early (F2/AGE-499): a decomposition demanding a stronger tier is rejected at submit_plan AND at the plan gate's readback, and an already-minted offender stage is parked BLOCKED at launch (the sweep holds tier-parked stages for a human) — never burned attempt-by-attempt against the attest-time ceiling. The Docker lane uses the stage worktree as its only writable host mount and --network none; E2E commands (test:e2e) run as runner-attested HOST commands under a small explicit env allowlist (D2 — the suite provisions its own isolated Docker database and cannot run inside the sealed container; the runner's tokens/keys are never in its environment). Because that sealed lane runs a tar copy of the worktree, and a stage checkout is a native git worktree whose .git is a gitdir pointer into the host repo, git verification commands can never attest there — working-tree-vs-HEAD is structurally unanswerable in the copy, image contents notwithstanding. F3 (AGE-648) rejects them at the same two points F2 uses: prove a diff-shape criterion in OPERATOR mode from the runner-attested DIFF_OR_PATCH artifact plus the EXECUTION_SUMMARY, or assert on file content instead. F2 checks the required tier; F3 checks whether the command can run in the lane at all. This matters more than it looks: stage verificationCommands are frozen at the decomposition mint and submit_plan on a stage job rewrites only the Work Order, so an infeasible command is unfixable from inside the stage's own job — it must never get minted. Evidence tiers form a total order — UNIT_MOCKED < INTEGRATION_LOCAL < PROD_TOPOLOGY_LOCAL < MANAGED_STAGING < PRODUCTION_DRILL (the middle value is retained for history) — and a mock stays valid at its declared tier: no test name or label upgrades it. Exit 0 is not proof that anything ran (W2): a suite whose cases were all skipped — a live-DB file with no database, a stray describe.skip — exits 0 and used to attest as clean evidence, with the counts living only in the executor's prose. A v2 attestation therefore also carries what the command's own output reported (counts: { testsRun, testsPassed, testsSkipped, suites }, parsed from the same bytes stdoutHash covers), and the gate refuses evidence that ran zero tests or whose tests were all skipped. A non-test command (typecheck, docs:check) attests counts: null — an explicit absence, never a fabricated zero — and a v1 attestation (no counts at all) stays tolerated, so in-flight evidence is never retro-invalidated. The attestation's version field stays 1 permanently: the payload revision rides attestationVersion, so an upgraded runner can never be rejected by a control plane that predates it. On a policy run the validation gate accepts TEST_EVIDENCE only when every declared command has a passing, current, sufficiently-tiered attestation; run-level acceptance additionally requires the evidence to be bound to the final subject (the final stage's diff digest or commit SHA — a later commit invalidates it). A stage that declares no commands requires no test evidence (honest absence over prose filler).

Bounded provenance-based review. A finding blocks by the SOURCE of its obligation, never its discovery round: on harness jobs a blocker (severity ≥ 60) must carry provenance — basisKind (BASELINE_REQUIREMENT, CHANGE_REGRESSION, EVIDENCE_INVALID, STANDING_INVARIANT are blocking-eligible; NEW_REQUIREMENT is change control, PRE_EXISTING_DEBT a follow-up, OPERATOR_OBLIGATION an acceptance item), a discoveryReason (after round 1 it must explain why the violation was previously masked — INITIAL_REVIEW cannot block late), a closure predicate, and the observed evidence. A rejection without a qualified blocker is coerced to APPROVED with the findings kept as follow-ups (the existing severity-gate downgrade, extended — not a second engine). Finding identity is a server-computed fingerprint (job + phase + basis + normalized title): a repeat reconciles/ reopens the existing row instead of duplicating it, and two reopens of one fingerprint park the loop for human adjudication via the round-cap marker. That count is kept on both reopen channels, because the rubric tells the reviewer to reopen the existing ledger row rather than re-file: submit_review bumps it when a repeat fingerprint reconciles, update_harness_finding bumps it when a finding moves back into OPEN, and re-projecting an M7 source never lowers the ledger's count. A rejection sustained by an already-twice-reopened ledger row parks for adjudication even when the reviewer submits no findings at all — without that the escalation was unreachable on the path the harness actually uses (M18 stage 6 rode one finding through four cap parks at reopenCount 0). Per-attempt workflow budgets remain configurable through maxIterations; the stage-wide reviewer budget is always six rounds, counted from the stage's newest cap park — a retry never resets it, but the human unblock a park demands re-arms it. Exhaustion parks to a human, never auto-approval; waived findings surface as CONDITIONAL, never as verified. The adversarial fixtures in src/server/harness/acceptance.benchmark.test.ts pin all of this: failing suites, mislabeled mocks, tier fraud, health-probe substitution, hidden operator actions, prose evidence, stale-SHA evidence, and late new-scope blocking are each rejected by the exact production predicate that guards them.

Delivery correlation

Once a stage commits, its branch flows into the delivery graph. The run's branch template puts the anchor task key first (harness/{taskKey}-{slug}-stage-{number}), so the inbound PR lands matched on the anchor task and the owning stage resolves by headBranch === HarnessStage.branch. Inbound PR/CI events — from both the GitHub webhook and the generic CI ingestion path — advance the stage's delivery dimension; a merged PR over a committed stage re-asserts the run's current-stage advance. These updates are system-sourced, atomically guarded (an in-transaction compare-and-set survives a concurrent branch reassignment or run archive), and never echo back out as a GitHub write or board webhook. A failing check is recorded (the harness Delivery tab + the registry), never auto-reverting the terminal committed state.

Connected sessions (M20.1)

A connected interactive session (CLI guide) may CORRELATE to an existing Agent Job/Run through the session's loose agentJobId/agentRunId links and display its phase — but the harness and workflow engines remain the only loop authorities: nothing a session does can advance, fork, or park a run by changing AgentSession.status, and session usage rollups never overwrite AgentRun economics. The session is an evidence anchor beside the loop, not a participant in it.

Control Tower queues

Stuck and frozen loops are visible and actionable as read-only, self-healing Control Tower queues over the harness spine (records, not fires — resolution routes through the existing operational-queue ledger):

QueueSurfaces
HARNESS_APPROVALSevery pending harness gate (spec/plan/commit), in exactly one queue
FROZEN_HARNESSESruns an operator has frozen
STUCK_HARNESSESrun-BLOCKED, stalled current stage, parked stages, aging gates, dead heartbeats, repeated CI failures, committed-but-unpushed stages (a COMMITTED stage on a linked repo whose branch never produced a PR — the runner likely crashed before brokering its push/PR credential), impossible states (the stage's newest execution job is DONE while the stage never reached COMMITTED/SKIPPED and no gate is pending — a dropped side-effect chain, flagged within ~a minute with the missing chain step named: gate unresolved / validation missing / commit boundary missing), and no-progress stalls (a progress-state stage with no state-changing activity for 30 minutes, no live agent turn, and no pending human gate)
HARNESS_OPEN_FINDINGSunresolved critical/high findings
HARNESS_REGISTRY_DRIFTa stored registry snapshot diverged from the projection
HARNESS_MISSING_EVIDENCEa stage short of its evidence requirement
HARNESS_LEASE_CONFLICTScontended stage/repo leases
ACTIVE_HARNESSESinformational, not an exception: one low-severity item per in-flight run — status, current stage n/m, whose turn is running — so the tower shows work in motion, not only work in trouble
UNLINKED_LEGACY_DATAM18.4 (R12): agent jobs, Work Orders, simulations, incidents, and playbooks that belong to no execution and to no enumerated standalone family. See Unlinked legacy data below

Every active queue is live-scoped (only in-flight runs), and a frozen or terminal run is never approved, rejected, or mutated through a queue.

Unlinked legacy data (R12)

R12 ("No abandoned paths") says every Work Order belongs to a Work Item and an execution step, every executable Agent Job belongs to an execution or an enumerated standalone family, every Simulation belongs to a dry-run execution, every Incident links to a Work Item / execution / deployment / release, and every Playbook either maps to a published harness template or is archived as legacy. The UNLINKED_LEGACY_DATA queue is where the exceptions surface.

Two rules decide what it shows, both in the pure classifier (src/server/control-tower/integrity-queue.ts):

  • The enumerated families are excluded, and the enumeration is closed. M7 review-loop jobs (request_agent_workflow → the base template), M8 delivery-workflow jobs (every registered delivery template key), the reference agents' webhook/cron dispatches (an automationKey prefixed bug-triage: / crm-follow-up: / standup:), and automation-dispatched jobs (any other durable automationKey). The delivery keys come from the SAME workflow-template registry requestDeliveryWorkflow validates against — a local copy is how a family silently stops being recognized.
  • Deterministically linkable records are backfilled, never queued. The cron sweep's backfillDeterministicLinks leg links what a UNIQUE key already decides — a job's stage → its run, a Work Order → its unique stage → that stage's run, an execution's template version → its template — with each write guarded on the column still being null. An ambiguous record is never guessed at; it stays in the queue for a human.

The queue stores nothing (the M3 never-a-store rule): a record leaves it by being linked or archived, so the next fetch simply stops emitting the item.

And it is complete, not merely correct. The SQL each kind runs is a candidate predicate, not the queue's predicate, so the source (sources/unlinked-legacy-data.ts) pages through the whole candidate stream per kind and classifies as it goes — no row bound between the read and the classification. A cap there would let rows the classifier then rules out (an M7 standalone job, a linked Work Order, a dry-run simulation) consume the page, and those are correct records nobody ever "fixes": they sit at the head of the table forever, so a genuinely unlinked record created after them could never be loaded and no operator action could surface it. That is the same defect a take on the lifecycle list was rejected for, and a bigger cap only moves the cliff. SCAN_PAGE bounds each round-trip and each IN (…) lookup — memory, never coverage — and the one cut that remains (INTEGRITY_ITEM_LIMIT items per kind) is applied LAST, over the classified set: everything it defers is a real orphan sitting behind other real orphans, and remedying the visible ones surfaces it on the next fetch.

Which is why the only actions it offers are the two that change the record (M18-AC20's "link/archive actions"), from one TOTAL table (INTEGRITY_REMEDY_TABLE):

RecordRemediesWhat the remedy does
Agent joblink, archivelink writes the execution link the sweep's backfill writes when a unique key decides it (the target is a harness run id, validated); archive archives the job as legacy — refused while it is still live work
Work OrderarchivearchiveWorkOrder. The execution step's workOrderId is a unique, harness-owned column written at decomposition, so an orphan plan is archived, never hand-assigned to a step
Simulationarchivea dry-run launch CREATES its simulation, so an orphan cannot be linked after the fact; refused while an APPROVED plan can still execute
IncidentlinkR12's execution arm through updateIncident, which refuses anything but a real EXECUTOR attempt in this workspace. Satisfying any other arm (Work Item, deployment, release) on the incident also clears the item
PlaybookarchiveR12's own remedy: "archived as legacy" (the archive also disables it)

There is deliberately no dismiss on this queue. A dismissal is stored in the Control Tower's resolution ledger and hides the item from every later read, so dismissing here would bury an unlinked, unarchived record while the integrity defect stayed in the database — the queue would report "all clear" about data it had merely been told to stop mentioning. Every remedy dispatches into the record's OWN ops core (where the authz, audit row and refusals live), the dispatch happens BEFORE the ledger row is written, and a refusal leaves the item open. The link remedy is the one resolution that carries a destination (targetId); every other queue's actions ignore it.

The observer projection (mirror board)

A harness run is not just governable — it is watchable in the product's own language. Once decomposition mints stages, the projection reconciler (src/server/harness/projection.ts) maintains an auto-managed mirror board named Harness — <run name> whose columns are the stage lifecycle: Queued → Planning → Ready → Executing → Review → Validated → Committed (+ Blocked). Each stage is a real task on that board (linked RELATES_TO the run's anchor task), and the card moves column-to-column as the stage transitions — through the ops core, so every move carries Activity (source system), realtime events, and shows up in the board UI live. The harness page links to the board; a stage's Work Order flips READY → LAUNCHED the moment its stage starts executing.

The reconciler is idempotent and self-healing: it runs post-response after any harness.* event and again from the cron sweep (which also backfills runs that predate the feature). Deleting mirror cards or columns just re-creates them; archiving the mirror board is the opt-out — an archived board is never resurrected. Agent liveness rides the same push: an agent whose linked token authenticated in the last 10 minutes shows ● active now in the registry.

Automations, webhooks, dashboard

  • Automations. Harness lifecycle events drive the existing rules engine (not a second engine) through seven harness.* triggers (harness.created, …stage_ready, …stage_validated, …stage_committed, …finding_created, …blocked, …completed) acting on the run's anchor task. The opt-in start_harness_stage_run action (harness triggers only — in practice harness.stage_ready) auto-launches the run's next stage, deduped against a running stage job. Harness automation actions run as source automation; a structural loop guard drops re-fired harness events so rules can never trigger rules.
  • Webhooks. Workspace-scoped outbound webhooks deliver the harness WorkspaceEvents (harness.created, harness.spec_validated, harness.plan_validated, harness.stage_committed, harness.frozen, harness.blocked, harness.completed, harness.stuck, …) using the same signed HMAC / retry / auto-disable / SSRF infrastructure as board webhooks. Harness payloads are compact ids plus a get_harness fetch hint only — no bodies, diffs, or free-form reasons cross the wire.
  • Dashboard. /[workspace]/dashboards/harness shows run lifecycle counts, stage throughput, the review-iteration histogram (every round, including clean ones), and the human-verification rate — one batched fetch + one pure aggregation, the same shape as the delivery dashboard.

Playbook mining

Once harness runs start completing, the daily outcomes cron also mines playbook drafts from them — sharing /api/cron/outcomes with the delivery miner and the DORA rollup. It clusters the HarnessRuns that reached COMPLETED over the trailing 90 days by branch strategy + board kind, and for each recurring pattern (at least two successful runs) upserts a playbook draft pre-filled with:

  • the harness loop tool surface it drives (the Phase-2 harness write tools plus the agent-workflow loop tools — the UI-first human gates are deliberately absent),
  • proposed eval cases derived from the loop (spec validated, plan covers every criterion, stages validate with full evidence, commit gate respected), and
  • the supporting AgentRun ids as evidence (capped per draft so the array stays bounded — the manifest records the cap, the true total, and the omitted count, so a bounded evidence set is never silent), plus the committed-stage total.

Drafts are deduped by (workspace, branch strategy + board kind) — the harness:… signatures are disjoint from the delivery miner's delivery:… keys, so both miners share the draft table without colliding. Pristine-only refresh: the miner only ever updates a draft that is still DRAFTED and has no human reviewer, enforced atomically (the refresh is a compare-and-set guarded by that pristine predicate, so a reviewer racing the cron is never clobbered). The moment a reviewer approves, rejects, or merely edits a draft, the signature is left untouched, so a re-mine can never clobber human-curated tools/eval cases or re-surface a decided pattern; an identical re-mine is skipped entirely so the cron never bumps updatedAt. Nothing is enabled automatically — a draft must be reviewed and then promoted by an admin behind the policy + eval coverage gate before it becomes a usable playbook.

Once promoted and enabled, a harness playbook feeds back into the loop: create_harness takes an optional playbookId that applies it to a new run — the playbook's mined branch strategy seeds the run's default (an explicit branchStrategy always wins), and the playbook is recorded as the run's provenance in the HARNESS_CREATED audit payload. Only harness-mined playbooks apply (a delivery-mined playbook is rejected — its patterns describe single delivery jobs, not runs), and only from the same workspace.

MCP surface

21 harness tools. Reads (read scope): list_harnesses, get_harness (the live registry projection — pending gates, active runs, and operability blockers are read through the companion calls listed in the runnable harness contract, not this tool; agent-role readiness has no read tool — see that section), list_harness_stages, get_harness_artifact, list_harness_findings. Writes (write scope): create_harness (optionally applying a promoted harness playbook via playbookId — see Playbook mining), submit_harness_artifact, submit_harness_review, update_harness_finding (one exception to the per-tool scope model: setting adjudication raises that call to admin scope + ADMIN role. The flag suppresses the reopen counter that drives the oscillation park, so ruling on a parked finding is an operator act, gated like freeze/cancel; a plain resolution stays at write so runner legs can keep resolving findings during revision rounds), request_harness_stage_run, submit_harness_stage_result. (The get_agent_job poll additionally carries a harness block on a stage execution job — run/stage ids, branch/repo/base + strategy/template, commit policy, and acceptance criteria — so the harness-aware runner detects and drives the stage without widening get_harness.) The generic submit_plan tool additionally accepts a harnessStages payload only for a decomposition job — the proposed ordered stages the accept gate mints (rejected for any other workflow). Admin (admin scope + role): freeze_harness, unfreeze_harness, cancel_harness, set_harness_policy, retry_harness_stage (relaunch-enable a stage parked BLOCKED by a FAILED or stalled execution job — runner crash, provider error, lapsed heartbeat: moves it BLOCKED → READY so request_harness_stage_run can open a fresh attempt with a fresh iteration budget, while the stage branch's working tree and prior findings carry forward. Only the run's current (first-undone) stage on an EXECUTING run; rejected for a stage blocked on an unresolved decision — accept the decision instead, so the decision gate is never bypassed. Each retry increments the stage's retryCount; on autonomous runs the sweep performs this retry automatically while retryCount is below the run's autoRetryLimit policy — see "Blocked-stage auto-retry" above), set_harness_agents (set the run's active PLANNER + REVIEWER agents so it can launch execution jobs — pass an AgentProfile id per role; the usable token is resolved server-side from the agent and never accepted from input; supersedes prior assignments by archiving them, and carries expectedUpdatedAt so two admins can't silently clobber each other), start_harness_authoring (move a DRAFT run into the authoring loop — DRAFT → AUTHORING, seed a run-owned draft Spec, and open the run-bound authoring agent job, emitting the normal agent.handoff; requires an anchor task, a repository, and active planner+reviewer agents; carries expectedUpdatedAt, and is not idempotent — a second start is rejected by the DRAFT guard. The run-owned spec is a SCAFFOLD seed pinned on the run so each run owns its own spec — two harnesses on one anchor task never share a draft; wiring the planner's reviewed output INTO that spec's versions is the harness-aware runner's job (H10.6). It only starts the loop; approving the plan-approval gate accepts the run-owned spec, pins its SpecVersion, and advances the run to PLANNING — resolved by a human in the Control Tower, with no MCP self-approval path), start_harness_decomposition (move a PLANNING run into the decomposition loop — open the run-bound decomposition agent job, emitting the normal agent.handoff; requires a validated/pinned spec, an anchor task, a repository, and active planner+reviewer agents; carries expectedUpdatedAt. The run stays PLANNING — it only starts the loop; the planner submits the proposed stages via submit_plan's harnessStages, and approving the plan-approval gate validates coverage and mints the ordered stages

  • 1:1 Work Orders, advancing the run to READY — resolved by a human in the Control Tower, with no MCP self-approval path. The generic request_agent_workflow path cannot start decomposition; this harness-bound tool is the only way in), create_harness_stage (hand-author a stage directly — the fallback for populating a run's stages outside the coverage-enforced spec→plan decomposition path; appends to the same ordered list request_harness_stage_run executes, carrying the same full stage plan, but bypasses coverage enforcement. admin scope + ADMIN role — authoring a stage changes the run's executable plan scope. A non-draft stage gets a 1:1 Work Order, auto-minted when workOrderId is omitted, so it has executable backing for request_harness_stage_run (the run must be task-anchored and non-terminal; a supplied workOrderId must back the run's anchor task). draftOnly: true creates a non-executable placeholder that request_harness_stage_run rejects, marked launchable: false in list_harness_stages / the UI registry), authorize_harness_local_commit (the external runner's pre-commit repo:commit permission check — keyed on the stage, returns { authorized, harnessRunId, branch }. Authorization only: it honors freeze / leases / repo:commit policy but mints no credential and approves no gate — the runner performs the local git commit itself, then reports the SHA via submit_harness_stage_result. The runner passes the resolved branch plus the changedPaths / diffLines the commit will touch, so branch-scoped leases and the high_risk_change diff-size policy evaluate the actual target before the commit. Only a VALIDATED/COMMIT_GATE stage on a non-frozen run can authorize; otherwise CONFLICT/FORBIDDEN. admin scope + ADMIN role — a run's own agent token can never reach it; the push/PR remains the separate single-use brokered repo:push / repo:pr_create path, and merge the separate always-human MERGE gate).

Creates take an idempotencyKey (replay returns the stored response; the same key with different args is a CONFLICT); update_harness_finding and set_harness_agents take an expectedUpdatedAt and return CONFLICT with the entity's current state on a stale write. set_harness_agents is also idempotent — re-applying the same planner/reviewer is a no-op (no new role rows, no audit) — but expectedUpdatedAt still guards a concurrent different replacement, since idempotency only protects an identical retry. Freeze/resume are admin operator tools (admin scope and role) — a run's own agent token can never reach them, and there is deliberately no MCP tool for an agent to approve its own gate, push, or merge. Mined playbook drafts are reviewed and promoted in the UI (the Playbooks page); only once an admin promotes one does it become a Playbook visible to the existing list_playbooks read tool — there is no MCP read for unpromoted drafts.

Concurrent execution & worktree isolation

Jentrix records and governs; the external reference runner and the managed Temporal harness worker do the actual git/build/test work. Historically both ran those Git operations in a single shared checkout, which is unsafe once two harnesses target the same repository — they can switch each other's branch, stage each other's files through git add -A, or contend on the index. The Concurrent Harness Execution hardening (M9/M12) makes one native Git worktree per execution attempt (AgentJob) the filesystem boundary, so parallel harnesses never share working state.

Product requirements: Concurrent Harness Execution PRD. Runner-operator setup lives in agents/README.md.

Per-AgentJob worktree isolation

The AgentJob id is the durable attempt identity already used by claims, Temporal workflow ids, stale-result fences, and idempotency keys, so it is also the worktree key — not the run or stage id, because a retry mints a fresh job that must not inherit the failed attempt's unreviewed files. Paths are deterministic:

<worktree-root>/<repo-key>/<job-id>/            ← the linked worktree (executor cwd)
<worktree-root>/<repo-key>/<job-id>.marker.json ← the SIBLING ownership marker

repo-key is a filesystem-safe encoding of the configured owner/name; job-id is validated as an opaque id and never interpreted as a path; every resolved path is proven to stay under the root after normalization and realpath/symlink resolution. The marker is a sibling (never inside the tree — the runner uses git add -A) and holds identifiers only (no paths, no secrets). The executor's working directory, diff/test capture, commit, and push all use this worktree; the reviewer receives it read-only. One shared, dependency-free helper (packages/execution/src/worktree.ts) does path derivation, create/adopt validation, superseded-branch detach, and safe cleanup for both the runner and the Temporal worker, so the two processes can never drift.

Harness reviewers that do not yet have a stage worktree use the repository's shared grounding checkout. Before every such reviewer turn the runner fetches the run's exact base branch, fast-forwards the checkout, and verifies that HEAD equals origin/<baseBranch>. A fetch, fast-forward, or equality failure parks the job with a clear grounding error instead of reviewing stale source.

Stacked ancestry — executionBaseRef

Sequential stages are a stacked series: a later stage builds on the previous stage's unmerged branch tip, not blindly on the run base, and the runner never resets to the checkout's mutable HEAD. The control plane resolves an executionBaseRef per stage (src/server/harness/base-ref.ts over the pure resolveExecutionBaseRef in src/server/harness/resolver.ts) by four rules:

  1. The target stage branch already exists (revision / restart / retry) → its own tip.
  2. First stage, no branch → the run's configured base branch (or the default base).
  3. Later stage whose predecessor is committed-but-not-merged → the predecessor's branch tip.
  4. Predecessor recorded merged into the run base (a MERGED GitHubPrLink) → the run base containing that merge.

Drift is measured, never silently tolerated. Stacking is correct, but a stage branch cut once and never rebased trails its base branch by however much lands there meanwhile — and that used to be invisible (while one M18 stage sat unmerged for two days, main took five harness fixes and the next stage planned against a stale CLAUDE.md). The runner measures git rev-list --count HEAD..<base> at claim time, tells every role turn the number in its git-truth context block, and records it on the attestation it mints; the job page renders "behind base by N" with a warning style at N > 0. Nothing auto-rebases — merging the predecessor or rebasing the branch stays an operator decision, and a pre-drift attestation reads as "not measured", never as "up to date".

The resolved ref is surfaced in the runner's stage context (and the get_agent_job harness block). Creating a fresh stage branch verifies the ref resolves to a commit first; an unresolvable/missing ref is an actionable MISSING_REF error (fetch only that ref through the brokered repository boundary), never a silent branch from HEAD.

Worktree lifecycle + external markers

  • Creategit worktree add on the stage branch from executionBaseRef, or check out the existing stage-branch tip on a retry; writes the sibling active marker.
  • Adopt — on restart, an existing directory is adopted only when Git metadata (inside-work-tree, top-level, checked-out branch, shared object store) and the sibling marker agree on repo/job/stage/branch; any mismatch, path escape, or symlink escape fails closed and touches nothing.
  • Bootstrap — after containment/ownership validation, the per-repo idempotent bootstrap argv run inside the worktree and allowlisted ignored config is copied (below). Idempotent because a crash may re-run bootstrap on adoption.
  • Checkpoint — a park must be lossless. An attempt that parks (failed turn, round-cap park, finding adjudication, killed runner) never reaches its commit gate, and the next attempt's worktree is cut from the stage branch's tip — so anything left uncommitted becomes invisible to every reviewer, gate, and operator (the M18 run needed ten manual "operator checkpoint" commits; one attempt left 2,881 lines behind). The runner therefore commits a parking attempt's dirty tree onto its stage branch as harness checkpoint: stage <n> job <id> (<reason>), and the same rescue runs at claim time — before a new attempt detaches a superseded worktree to reuse its branch. The guard is the branch ref, not the commit: a commit in a detached worktree succeeds and advances nothing, so the checkpoint refuses unless the branch is attached and re-reads refs/heads/<branch> to prove it moved. A clean tree commits nothing. When the work cannot reach the branch, the claim is refused with the worktree path named for a human — parking loudly beats overwriting it silently. A frozen run is never checkpointed (a local commit is a freeze boundary, and the same job re-adopts its worktree on resume).
  • Detach — a retry legitimately requests the same stage branch in a new worktree; Git refuses while the old worktree still holds it. Only after fresh Jentrix state proves the old attempt superseded (its lease was taken over, or a newer stage job replaced it and the old job has no active run or pending commit) does the runner git checkout --detach the old worktree — preserving all dirty files — and stamp its marker superseded-detached. Live or unprovable ownership fails closed, naming the owning branch/job; nothing is reset, renamed, or stolen.
  • Retain — worktrees survive freeze, the PRE_EXECUTION/FINAL_ACCEPTANCE/COMMIT gates, same-job revisions, and a process restart; a source-dirty (tracked changes or non-ignored untracked files) or orphaned worktree is retained and reported, never force-removed.
  • Safe-remove — only on a safe terminal Jentrix state with no active run: git worktree prune → validate containment + marker → git status --porcelain classification → for clean/ignored-only, git clean -fdX (disposable bootstrap output only), then git worktree remove (no --force) and delete the marker. It never recursively deletes an arbitrary path.

Runner configuration (env)

STACKS_RUNNER_REPOS maps owner/name → a local checkout. It stays back-compatible with string values and also accepts an operator object:

{
  "owner/name": {
    "cwd": "/code/name",
    "bootstrap": [["pnpm", "install", "--frozen-lockfile"]],
    "copyIgnoredFiles": [".env.local"],
    "worktreeRoot": "/var/stacks-worktrees"
  }
}
  • bootstrap is argv arrays (never interpolated shell), run after validation, idempotent. For Jentrix, pnpm install --frozen-lockfile suffices — its shared content-addressed store links node_modules per worktree and postinstall runs prisma generate.
  • copyIgnoredFiles copies only repo-relative files that git check-ignore confirms are ignored, with owner-only permissions, never logged/artifacted, and deleted with the worktree. Nothing is copied by default; operators may inject env into the subprocess instead.
  • STACKS_RUNNER_WORKTREE_ROOT selects the worktree parent directory (a per-repo worktreeRoot overrides it); when omitted, a sibling .stacks-worktrees directory next to the checkout is used.
  • STACKS_RUNNER_MAX_CONCURRENT_JOBS bounds planner/reviewer/executor agent turns — default 1 (serialized, prior behavior), hard cap 16.
  • STACKS_RUNNER_MAX_CONCURRENT_COMMITS bounds post-validation commit/push continuations on a dedicated pool, so a freshly-approved commit never queues behind a long model turn — default 1, hard cap 8.

Each value parses to a finite maximum (unset / non-integer / < 1 → default, never unbounded; above the cap → clamped down). The webhook and poll inputs share one queued/running job-id dedup registry across both pools, so a duplicate delivery collapses before consuming a slot; claim_agent_job's server-side CAS stays the final ownership guard.

Operator worktree listing

Before any manual cleanup, list runner-owned worktrees:

cd agents && pnpm harness-worktrees

It shells git worktree list + reads ownership markers (read-only, never touches Jentrix) and prints one row per worktree with repo / job / stage / branch / age / source-cleanliness and the absolute local path — to stdout for a human only, never persisted into Jentrix activity or artifacts. Source-dirty, superseded-detached, and orphaned worktrees are flagged with a manual-cleanup instruction; the automatic cleanup path only removes a fully-delivered clean worktree.

Temporal parity

The Temporal worker resolves repo context per workflow input, not from process-wide env, so multiple stage workflows share one task queue without sharing checkout state. HarnessStageWorkflowInput carries the immutable repo identity (owner/name), base branch, resolved executionBaseRef, and resolved head branch; the dispatcher stamps them (and throws before workflow.start if it can't fill them, so a new workflow never silently lands on the shared checkout), and a Git-capable Activity resolves the owner/name through the worker's trusted repo map (STACKS_HARNESS_REPOS, same shape as STACKS_RUNNER_REPOS; an unmapped repo throws in a pure function before any filesystem/network effect) and derives its worktree via the shared helper.

One-generation fallback. A pre-migration in-flight workflow replays with no repo fields; for exactly one compatibility deployment such an Activity falls back to the process-wide STACKS_HARNESS_REPO_CWD / _GITHUB_OWNER / _GITHUB_REPO / _GIT_HEAD settings and emits a harness.temporal.per_run_fallback compatibility metric. Newly dispatched workflows never use it; after pre-migration workflows drain, a later deployment makes the fields required and removes the fallback.

Worker concurrency env. The worker sets concurrency explicitly instead of relying on SDK defaults: STACKS_WORKER_MAX_CONCURRENT_WORKFLOW_TASKS (default 40) and STACKS_WORKER_MAX_CONCURRENT_ACTIVITIES (default 1). Activity concurrency > 1 is refused unless STACKS_WORKER_GIT_WORKTREE_ISOLATION=true attests that every dispatched workflow carries per-workflow repo context (the fallback is drained) — the fail-closed gate against concurrent commits corrupting a shared checkout.

Acceptance sweep (PRD criteria → stage → test)

Every acceptance criterion in the Concurrent Harness Execution PRD maps to the shipped stage and the test that exercises it. No criterion is orphaned.

#Criterion (abbreviated)StageTest
1Two same-repo jobs run concurrently at MAX_CONCURRENT_JOBS=2, each its own worktree/filesS4 (+S3)agents/runner-pool.smoke.test.ts "bounds concurrency… a third same-repo job waits"; agents/harness-worktree.smoke.test.ts
2Distinct commits on resolved branches, no shared-index / branch-switch raceS3 / S4agents/harness-worktree.smoke.test.ts; agents/runner-pool.smoke.test.ts
3A third job waits; observed concurrency never exceeds the boundS4agents/runner-pool.smoke.test.ts "bounds concurrency to the configured max; a third same-repo job waits for a slot"
4Duplicate webhook + poll dispatches → one claimed turn / one worktreeS4agents/runner-pool.smoke.test.ts "one shared registry collapses duplicate webhook + poll deliveries"
5Kill after edits before submission, restart → adopt, no lost/duplicated changesS1 / S3packages/execution/test/worktree.test.ts "adopt succeeds on marker+git agreement"; agents/harness-worktree.smoke.test.ts "…a same-job restart adopts"
6Worktree survives freeze, PRE_EXECUTION / FINAL_ACCEPTANCE / COMMIT gates, revisionsS3agents/harness-worktree.smoke.test.ts (retain path); packages/execution/test/worktree.test.ts cleanup gating
7Stage 2 begins from stage 1's unmerged tip; when merged, from the run baseS2src/server/harness/resolver.test.ts "resolveExecutionBaseRef" rules (3) committed-but-not-merged & (4) merged-into-base
8Fresh retry job → different worktree, same branch; superseded detached with files preserved; late submits rejectedS3agents/harness-worktree.smoke.test.ts "a CONFIRMED-SUPERSEDED owner → detach old (files preserved) + reuse branch"
9Branch collision with a live / unprovable attempt (incl. null/legacy/default-template) rejected, owner named; nothing stolenS3agents/harness-worktree.smoke.test.ts "a LIVE owner → conflict; the old worktree/branch is untouched"; "…NO runner marker is UNPROVABLE (never stolen)"
10Real-project worktree finishes deps install, Prisma generate, ignored env provisioning before the executorS3agents/harness-worktree.smoke.test.ts "bootstrap runs in the worktree after validation"; "copyIgnoredFiles: only allowlisted…"
11Ignored-only worktree cleaned/removed; tracked / non-ignored changes retained + reportedS1 / S3packages/execution/test/worktree.test.ts "cleanup: ignored-only output…" / "cleanup: source-dirty… retained"; agents/harness-worktree.smoke.test.ts "cleanup removes a clean worktree, retains a source-dirty one"
12Approved commit continuation starts on its dedicated pool while the agent-turn pool is fullS4agents/runner-pool.smoke.test.ts "the dedicated commit pool runs concurrently with a saturated agent-turn pool"
13Two Temporal workflows on the same repo complete concurrently with distinct worktrees + correct base/headS5services/worker/test/harness-repo-context.test.ts "new path resolves the worktree config from the trusted map"; worker-concurrency.test.ts "allows activity concurrency > 1 once isolation is attested"
14Pre-migration workflow (no repo/base/head) completes via fallback; new workflows never use itS5services/worker/test/harness-repo-context.test.ts "no repo identity ⇒ fallback to the process-wide checkout env"; "THROWS before dispatch when any required field is missing (never emits {})"
15Temporal replay stays deterministic; crash / idempotency tests greenS5services/worker/test/harness-crash-contract.test.ts, harness-stage.workflow.test.ts (existing replay/crash suites) alongside harness-repo-context.test.ts
16At concurrency 1, no behavioral regression despite the new worktree cwdS3agents/harness-runner.smoke.test.ts, agents/execution-parity.smoke.test.ts (run at the default MAX_CONCURRENT_JOBS=1)

Phase 4 (operational hardening / canary) is an operator follow-up, out of scope for these code stages. Enabling a production concurrency value > 1 on a canary runner and watching queue / disk / branch-conflict / cleanup metrics — then raising concurrency only after the same-repo acceptance probe stays green — is the PRD's Rollout Phase 4 and belongs to operations, not the S1–S5 implementation.

Bootstrapping a project with a harness

Set up your first project gives a coding agent one prompt to analyze a repository and create a board, project, and anchor task. These are the extension clauses that add a governed harness on top — paste them into that prompt. They moved here because a harness is a full-deployment capability and the base prompt has to stay true on any deployment.

Extra prerequisites

  • Register two active agent profiles — a planner and an independent reviewer — and link exactly one usable PAT to each.
  • Bring the external runner online before asking Jentrix to start authoring. You can skip this and still create a safe draft harness.
  • The credential doing the bootstrap needs admin in addition to read and write: harness configuration is role-gated, and scopes are not hierarchical.

Extra hints

- Create a harness: [yes]
- Start harness authoring now: [no]

Extra proposal clause (Phase 1, alongside E)

   - when requested, one DRAFT harness linked to the project and anchor task,
     using the `owner/name` git remote, the detected base branch, PER_STAGE
     branches, sequential execution, an independent reviewer, and only
     evidence-backed allowed paths. Use an unrestricted path list when the
     repository does not provide a trustworthy boundary—do not guess globs;
   - for a first harness, propose `requireHumanGates: true`,
     `autoAdvanceStages: false`, and `autoRetryLimit: 0` so I can observe every
     boundary before opting into autonomy.

Phase 1 discovery should also read agent profiles, harnesses, and GitHub connection state, and Phase 1's summary may note repository boundaries that could become harness allowedPaths.

Extra creation clauses (Phase 2, after I)

J. If approved, create the harness linked directly to the project and anchor
   task. Apply the first-run policy above. Re-read the harness before any
   update-style call and pass its current `updatedAt` as the optimistic
   concurrency value when the command supports it.
K. Inspect the eligible agent profiles. Assign a planner and a different
   reviewer only when both are ACTIVE, allowed in this workspace/project, and
   each has exactly one usable linked PAT with the needed scopes. Otherwise
   leave the harness in DRAFT and report the exact Agents-settings work needed.
L. Start authoring only when the hint says yes, the objective is confirmed, two
   eligible agents are assigned, and the external runner is online. Starting
   authoring opens a real agent job. Otherwise leave a configured DRAFT harness.
   Do not start decomposition; that comes only after the authored spec is
   reviewed and its gate is approved.

The final validation clause should then also report the harness status and policy, and include agents, GitHub, and gates in its UI checklist.

Why the first harness is deliberately manual

Human gates, manual stage advance, and zero automatic retries let you inspect the authored spec, decomposition, evidence, and commit boundary. After one successful run, change those settings from the harness Settings tab or with set_harness_policy.

The prompt leaves the harness in DRAFT unless the execution path is actually ready. Creating a harness does not run an agent, and Jentrix itself does not edit the repository. start_harness_authoring opens the first agent job; the external runner does the work, and the lifecycle from that point is the runnable-harness contract.

Afterwards, open the draft harness, confirm its planner/reviewer and first-run policy, and start authoring when the runner is available. You can also add a harness later from an existing task's Delivery section, without rebuilding the project structure.

Running the runner in production

Moved here from the deployment guide: this is harness-runner operation, not app deployment. The app itself needs none of it.

Runner authentication (no standing PATs)

The harness/review-loop runner (agents/workflow-runner.ts, run on an always-on host — it does not deploy to Vercel) authenticates its three principals — runner (read,write,admin), planner (read,write), reviewer (read,write) — through a mode-selected provider. As of M12.4b (AGE-174) the managed execution path carries zero standing PATs; tokens are short-lived and refreshed at every drive. Select the mode with STACKS_RUNNER_MODE:

ModeWhoAuthEnv
managedthe Jentrix-operated planeWorkload identity + per-run capability tokens (M12.4a). The worker holds a short-lived, auto-rotated SVID and trades it at the control-plane token exchange for run/workspace/role-scoped capability tokens (each only ever a narrowing of the run's P2.1 authorization). No standing PAT — the provider fails closed if one is present in its env.STACKS_TOKEN_EXCHANGE_URL, STACKS_WORKLOAD_SVID, STACKS_RUN_ID, STACKS_WORKSPACE_ID
self-hosteda runner you operateM11.1 bootstrap + claim-URL flow. A one-time STACKS_BOOTSTRAP_TOKEN is redeemed at a single-use, short-TTL claim URL for short-lived role tokens; it is spent exactly once and then refreshed with a rotating refresh credential. No standing PAT required.STACKS_BOOTSTRAP_TOKEN, STACKS_CLAIM_URL
(unset, legacy PATs present)backward compatibilityDEPRECATED. Reads the standing STACKS_RUNNER_TOKEN / STACKS_PLANNER_TOKEN / STACKS_REVIEWER_TOKEN and logs a migration notice at startup.the three PATs

For self-hosted, a workspace admin creates the one-time credential through POST /api/agents/bootstrap with a bearer that has admin scope and JSON { workspaceId, plannerAgentId, reviewerAgentId }. The response contains the one-time bootstrapToken, its claimUrl, and both expiries. Each agent needs exactly one non-revoked linked API-token row; it may be expired because the row is only a stable rotation anchor. Issuance invalidates any old raw bearers immediately; claims rotate those rows in place so existing workflow turn-token ids remain valid. Bootstrap expires after 15 minutes, role tokens after one hour, and the rotating refresh session after 24 hours. See the command in agents/README.md.

Migrate self-hosted runners to STACKS_RUNNER_MODE=self-hosted. The standing-PAT env vars are deprecated and the managed plane refuses them: a grep test (agents/runner-auth.smoke.test.ts) fails if any managed-path module even names one, and the managed provider throws StandingPatOnManagedPathError if a standing PAT leaks into its environment. Full setup lives in agents/README.md.

Each harness execution attempt uses its own native Git worktree. Configure the trusted source checkout map in STACKS_RUNNER_REPOS; optionally set STACKS_RUNNER_WORKTREE_ROOT. Agent turns and commit continuations use separate bounded pools controlled by STACKS_RUNNER_MAX_CONCURRENT_JOBS and STACKS_RUNNER_MAX_CONCURRENT_COMMITS (both default 1). Increase them only after sizing model, CPU, memory, and disk capacity for the runner host.

Managed execution plane

The M12 execution plane is deployed separately from Vercel: a Temporal worker drives durable workflows and the sandbox controller runs isolated workloads. The production topology, secret-store boundaries, Kubernetes manifests, revision compatibility checks, rollback procedure, and staging evidence commands live in the deploy runbook; local Temporal parity and worker operation are covered by the worker guide.

The worker reads TEMPORAL_ADDRESS, TEMPORAL_NAMESPACE, and TEMPORAL_WORKER_API_KEY. The control plane derives its own stamps and needs no Vercel env var: STACKS_SCHEMA_REVISION falls back to src/server/revision/schema-revision.generated.ts (regenerated by pnpm gen:schema-revision, drift-gated in CI, compiled into the build) and the app revision comes from VERCEL_GIT_COMMIT_SHA. Stamp the separately-built worker and sandbox images to match what GET /api/health reports under revision.control; managed claims fail closed when the recorded schema revisions are incompatible. Setting STACKS_SCHEMA_REVISION explicitly overrides the derivation on any component — that is the rollback pin, and a forgotten one freezes the stamp. The checked-in deploy/ configuration is a template: replace its Temporal Cloud and cluster placeholders through the deployment secret/config system before bring-up.

Documentation commit guard

A public-contract change must land its GENERATED doc entry in the SAME commit (Architecture decision 9). The canonical case: an MCP tool added without its generated docs/agent-platform.md catalog row and tool-registry.generated.json manifest entry. One checker enforces this at three boundaries; it only ever checks — regeneration is the separate pnpm docs:sync.

  • pnpm docs:sync regenerates the checked-in generated docs (the tool registry + docs catalog) from the live MCP surface. pnpm docs:check fails when a public contract is stale, naming the exact document(s) + the fix command and nothing else — an internal-only refactor (no src/lib/mcp/** / docs/agent-platform.md / tool-registry.generated.json change) passes without running the regeneration. Scope flags: --staged (pre-commit), --base <ref> (CI merge-base range), none (full drift check, used by the managed harness).
  • Local: the tracked .githooks/pre-commit runs pnpm docs:check --staged. postinstall (scripts/install-git-hooks.mjs) points git config core.hooksPath at .githooksno Husky / lint-staged dependency, just native git.
  • CI: the ci job runs the same pnpm docs:check --base <sha> over the pushed range (github.event.before; PRs fall back to origin/main).
  • Managed harness: the runner runs docs:sync before the final review turn (so the regenerated files are in the diff the reviewer sees — never silently synced-and-staged behind review) and docs:check immediately before authorize_harness_local_commit. A failing check hands its outcome + a checksum of the change to the authorize tool, which records a checksum-bound DOCUMENTATION_REVIEW finding (category: DOCS, targetType: documentation_review, targetRef: <diff checksum> — no new column) and BLOCKS the local commit. A human may resolve the finding as docs not required (ACCEPTED) for that exact diff; any subsequent diff (a different checksum) invalidates that resolution and re-opens a fresh review. Server logic lives in src/server/harness/docs-review.ts (recordDocumentationReview), enforced in authorizeLocalCommit (src/server/harness/commit.ts).

Security recap

  • Push / merge / deploy / rollback and credential grants are policy-gated + brokered (single-use, scoped, short-lived) + leased — an agent never holds a standing raw repo or cloud token. Local commit is runner-performed under a repo:commit check + SHA/diff report.
  • No secrets, raw transcripts, or unrestricted credentials in prompts, comments, Work Orders, or artifacts — redactSecrets runs over every stored log, artifact, and Runbook body on ingest.
  • Freeze is enforced at every Jentrix boundary (claim/submit reject or park, heartbeat returns frozen, brokered grants fail closed); the runner is contractually required to pause on its next heartbeat. Resume is attributed and audited.
  • Every harness mutation funnels through an authorization guard, writes its audit row in the same transaction, and carries its actor source (agent, automation, or system).