[developer-review-engine-proof-v1] 7 — An Engine run in shadow mode does the real work and makes zero forge effects #1648

Activity

1. daniboomerang

   added this to the Productionize developer-review on Atta Engine milestone on Oct 25, 2026
2. daniboomerang

   added

   vinaya/tranche:developer-review-engine-proof-v1Tranche: developer-review-engine-proof-v1Tranche: developer-review-engine-proof-v1

   on Oct 25, 2026
3. daniboomerang commented on Oct 25, 2026

   

   daniboomerang

   on Oct 25, 2026

   MemberAuthor

   More actions

   Brief hash: 4db24833b3aac3a15c076f166797c1d204cce44ab5885bf799a00c393d61b91b
   For: [model] (coding-agent CLI on a dev machine, dispatched locally, unattended)
   Reason: Suggested agent-class — high: the classifier must cover every write path, which needs judgment over many functions.
   Owner: the Principal
   Goal: An Engine run in shadow mode does the real work and makes zero forge effects
   Project: cli
   Tier: 1

   Closes #1648

   You are the AEG Developer. Run bun apps/cli/src/index.ts doctrine --role developer --print and read its output first. Mandatory.

   Objectives

   O1. An Engine run started in shadow mode makes zero forge effects: across every scenario of the behavioural corpus, run in shadow mode with the Claude stub and with the Codex stub, the forge-fake ledger records no commit, push, pull-request open, published round or marked comment, while the same scenarios run without shadow mode still record their expected effects.
   O2. Shadow mode is mechanically unable to mutate: every forge-writing function the Engine actions can call and every Developer publication tool is replaced by a recorder that refuses and records the attempt, and a test fails when a forge-writing function or a Developer tool exists that the shadow replacement does not classify.
   O3. A shadow run still runs the real work around the effects: the Developer and both reviewers are dispatched, the gates run, and the run's recorded decision and its refused-attempt list are written to the shadow run's own runtime directory, never to the task's live runtime directory.
   O4. The Engine runner script accepts a shadow flag that starts a shadow run of a real task beside its live run, with its own runtime directory and its own copy of the worktree, and refuses to start when that directory is the task's live runtime directory.
   O5. The shadow-mode fault-and-attack row and cutover gate 9 name this evidence, and the exit check reports no owed Engine evidence for them.

   Documentation

   • apps/cli/specs/dev-review-evaluation-program.md — §6.13 governed effects (shadow mode is mechanically unable to mutate; zero effects in shadow mode), §13 row "Shadow-mode mutation", §12 gate 9 (O1, O2, O5)
   • processes/vinaya/executing/actions.ts — BoundFunctions and ACTION_BINDINGS, the forge-writing functions every Engine action reaches through task.bound (O2)
   • apps/cli/src/lib/task-tools/dev-tools-server.ts — DevToolContext, the Developer's publication tools (O2)
   • processes/vinaya/bin/run-engine.ts — EngineRunOptions (actions, handoffDeps, devToolsContext) and parseArgs (O3, O4)
   • apps/cli/tests/conformance/dev-review-engine-loop-adapter.ts — the Engine scenario drivers and the forge.jsonl effect ledger they read (O1)

   Premises

   Checked against the code when this task Issue was cut, and again against the default branch at dispatch — facts, not claims.

   • processes/vinaya/executing/actions.ts contains export interface BoundFunctions {
   • processes/vinaya/executing/actions.ts contains bound?: Partial<BoundFunctions>
   • apps/cli/src/lib/task-tools/dev-tools-server.ts contains export type DevToolContext = {
   • processes/vinaya/bin/run-engine.ts contains handoffDeps?: HandoffDeps
   • apps/cli/tests/conformance/dev-review-engine-loop-adapter.ts contains forge.jsonl
   • apps/cli/tests/eval/registry/shards/fault-attack.json contains fault-attack:29-shadow-mode-mutation

   2. Context — read before doing anything

   • Tranche: developer-review-engine-proof-v1, task 7, Issue [developer-review-engine-proof-v1] 7 — An Engine run in shadow mode does the real work and makes zero forge effects #1648. Branch task/developer-review-engine-proof-v1/7. Depends-on: #1643, Conflicts-with: —.
   • Read Issue [developer-review-engine-proof-v1] 7 — An Engine run in shadow mode does the real work and makes zero forge effects #1648 in full for the complete rationale — do not re-derive it.
   • Revision: rendered at 8d5fc58a61e2d1daeed5aa58f48e7212bbedbb76 — the checkout's HEAD equaled the remote default branch, and no pinned file below carried an uncommitted change, when these facts were read.
   • Boundary — Add a shadow mode to the Engine runner: one module classifies every forge-writing bound function and every Developer publication tool and builds refusing recorders for them; the runner script gains a shadow flag; a conformance test runs the whole corpus in shadow mode on both stub vendors and asserts a zero-effect ledger. It does NOT compare shadow results with standalone results beyond effects (difference classification is later canary work), does NOT add shadow mode to the standalone loop, and does NOT change any live effect path. Pinned files: processes/vinaya/executing/shadow.ts, processes/vinaya/bin/run-engine.ts, processes/vinaya/tests/shadow.test.ts, apps/cli/tests/conformance/dev-review-engine-loop-adapter.ts, apps/cli/tests/conformance/dev-review-engine-shadow.test.ts, apps/cli/tests/ci-shards/shard-3.txt, apps/cli/tests/eval/registry/shards/fault-attack.json, apps/cli/tests/eval/registry/shards/cutover-gates.json. Out: the CLI library is in the Surface only because the task-run Engine module imports the runner script, and it is not edited; the standalone loop, the effect executor, the Operator tools, every live forge writer.
   • Traps to avoid — Do NOT implement shadow mode as a flag the live actions check before writing; that leaves the live writer reachable. Build the shadow run from refusing recorders so no live forge writer is in the registry at all. Do NOT let an unclassified BoundFunctions key or DevToolContext method pass silently; the classification test must fail on a new key. Do NOT share the live run's runtime directory, lock or worktree; a shadow run beside a live run must not take its driver lock. Do NOT write Issue or pull-request numbers or tranche slugs into code comments. A new test file under apps/cli/tests needs its line in one shard file in the same commit.

   reader-resolvable-prose: code comments, READMEs and specs carry no Issue or pull-request numbers and no tranche slugs.

   doc-claims: every AEG:CLAIM line in a changed file is kept true in the same commit.

   atta-labs/process-version-bump: the task leaves processes/vinaya/process.yaml, the flow file, the role files and processes/vinaya/executing/actions.ts unchanged, so the check passes as skipped; if actions.ts must change, process.yaml's version is raised in the same commit.

   3. Technical dependencies

   Dependency rationale — Depends-on: #1643 — task 2 adds the per-backend evidence field to the registry rows and the exit check's owed-evidence list, which O5 fills for the shadow-mode row and gate 9. Shares apps/cli/tests/eval/registry/shards/fault-attack.json with #1643–#1647 and #1649; processes/vinaya/bin/run-engine.ts with #1645 and #1647; apps/cli/tests/eval/registry/shards/cutover-gates.json with #1643 and #1650; apps/cli/tests/conformance/dev-review-engine-loop-adapter.ts with #1649 and #1650; and apps/cli/tests/ci-shards/shard-3.txt with #1642 and #1649. Only #1649 reaches three shared files (fault-attack.json, the loop adapter, shard-3.txt), and #1649 declares Conflicts-with on this task; every other sibling shares two files at most, under the collision threshold.

   4. Technical surface map

   Create:

   • (none — every surface file already exists)

   Modify:

   • processes/vinaya
   • apps/cli/tests
   • apps/cli/src/lib

   Out of surface: harness, packages, aeg-root — a change there, or to any path no Surface in: glob covers, is still published: the review-loop driver records each such path in the pull request body's beyond-Surface block, and the reviewers judge it like any change.

   Premise pins

   Premise:

   • apps/cli/tests/ci-shards/shard-3.txt sha256: 771fcba3f1e28e6dda03ed63235c6207620702866a5af83cef293d933ce468f2
   • apps/cli/tests/conformance/dev-review-engine-loop-adapter.ts sha256: 28e342d6ed7d2c9545a913d065dde0e803cc39651148959fd6aa2c4cd7383ee5
   • apps/cli/tests/eval/registry/shards/cutover-gates.json sha256: 8c95becafe7325d5614e7fbd8d730450983d1da83a04bff836c44e897bd44bc4
   • apps/cli/tests/eval/registry/shards/fault-attack.json sha256: 61e80e09fad63fb1cf990f6748fbc286fd563cf8ab4a4bb1d1b6640fec376be6
   • processes/vinaya/bin/run-engine.ts sha256: 5e918f14b6f473e114dec98da5c94e4c8b8bf6576149b452e947c7bd9a4d773e

   5. Pre-flight checks

   Step 0 (mandatory, verbatim):

   bun install --frozen-lockfile --silent
   

   1. Clean status; parent origin/main; branch suffix literal-matches the task id.
   2. Under the review loop (bun apps/cli/src/index.ts task run), the driver has already run the dispatch-readiness gate plus, on this repository toolchain, its own fuller derivation from its own unsandboxed process before dispatching you (re-derived at render time: it was ready), and dispatches you only when that verdict is READY — a NOT READY verdict refuses the turn before it starts, so being dispatched is the verdict: look for no file and run neither script, whose own gh call fails inside your sandbox. Working manually, with no driver behind you, fall back to roles/developer.md's own items 3/5/7 prose — each a complete, hand-run forge check in its own right.

   On any failure: STOP and report.

   6. Numbered parts — publish once with publish_changes when every Part is done, then open_pull_request

   1. Part 1 (O2): The shadow classification of every forge-writing bound function and Developer tool, and the refusing recorders built from it.

      Files (touches @attalabs/vinaya):

      • apps/cli/tests/ci-shards/shard-3.txt
      • apps/cli/tests/conformance/dev-review-engine-loop-adapter.ts
      • apps/cli/tests/eval/registry/shards/cutover-gates.json
      • apps/cli/tests/eval/registry/shards/fault-attack.json

      The pre-push hook runs the affected suite on your one push and refuses it on failure — do not run it yourself per Part.

   2. Part 2 (O3, O4): The runner's shadow flag, its own runtime directory and worktree copy, and the refusal when it would reuse the live directory.

      Files (touches the repo root):

      • processes/vinaya/bin/run-engine.ts

      The pre-push hook runs the affected suite on your one push and refuses it on failure — do not run it yourself per Part.

   3. Part 3 (O1): The corpus run in shadow mode on both stub vendors with a zero-effect ledger, beside the non-shadow run with its expected effects.

   4. Part 4 (O5): The shadow-mode row and gate 9 name this evidence.

   7. Documentation-update list

   .vinaya/doc-owners derivation matched zero bindings against this task's surface — no mechanically-derived doc updates. Confirm no additional doc artifact applies before treating this list as final.

   8. Verification before claiming done

   • Under the review loop, the controller runs the static gates: the commit hook applies safe format and lint fixes to staged files, typechecks affected packages, and runs the Vinaya checks; the pre-push hook runs the affected suite; run_checks and CI provide the authoritative results. Do not hand-run or paste static-gate commands. Working manually, run this repository's declared typecheck, lint/format, build, and documentation-gate commands yourself.
   • The pre-push hook runs the affected suite once, on your one push, and refuses it on failure; CI runs the full suite.
   • Every blast-radius consumer named in §4, re-verified by name.
   • The tier checklist in bun apps/cli/src/index.ts doctrine --role developer --print genuinely satisfied; use the controller's run_checks result, hooks, and CI as gate evidence.

   9. Test Plan

   cd processes/vinaya && bun test --timeout=30000 tests/shadow.test.ts → 0 fail; every bound function and Developer tool is classified; an unclassified key fails
   cd apps/cli && bun test --timeout=60000 tests/conformance/dev-review-engine-shadow.test.ts → 0 fail; every corpus scenario on claude and codex records zero ledger effects in shadow mode
   cd apps/cli && bun test --timeout=30000 tests/eval/exit-check.test.ts tests/ci-shards.test.ts → 0 fail
   bunx turbo run typecheck --affected → exits 0
   

   10. Stop conditions

   • An Engine action writes to the forge through a function outside task.bound, the handoff dependencies and the Developer tool context.
   • The corpus driver cannot inject the shadow task input without changing the scenario fixture.

   Stop-and-escalate — Stop if an Engine action writes to the forge through a function not reachable from task.bound, the handoff dependencies or the Developer tool context; name the action and the function. Stop if the corpus driver cannot inject the shadow task input without changing the scenario fixture; name the scenario.

   11. Constraints

   Autonomy: Do not stop to ask clarifying questions. For any ambiguity not covered by a Section 10 stop condition, choose the most reasonable option consistent with this brief, record the choice in the PR body at open, or in a PR comment after open, and continue. Halt only for the explicit Section 10 stop conditions — and when you halt, record the blocker in a PR comment or an Issue comment rather than waiting interactively for input.

   12. Deliverable

   • PR title (exact): [developer-review-engine-proof-v1] 7 — An Engine run in shadow mode does the real work and makes zero forge effects
   • Open the PR only via the open_pull_request tool (title and body); publish your commits with publish_changes and update the body with update_pull_request_body/refresh_evidence — you hold no gh/git push credential.
   • Publish once, with publish_changes, when every Part is done; then open the pull request with open_pull_request. Publish again only to answer review findings or a red gate — a refused hook, a failing check, a merge conflict.
   • PR body = the Developer's PR report (print it with bun apps/cli/src/index.ts doctrine --template pr-report --print), with Closes #1648 at the top of the header block. This brief stays frozen on the task Issue and never rides in the body.
   • Pre-open: tier checklist satisfied. The documentation gate is the controller's — run_checks and CI run it.
   • Include git diff origin/main...HEAD --stat as context only; Surface is judged by the driver's surface-scope check, never by this hand-run diff. Token use needs no report — the Vinaya log records it.
   • Then STOP. Review and Verification are separate invocations.
4. daniboomerang

   self-assigned this

   on Oct 25, 2026
5. daniboomerang

   closed this as completedin #1664on Oct 25, 2026